[Met_help] [rt.rap.ucar.edu #89704] History for Python embedding for grib format
John Halley Gotway via RT
met_help at ucar.edu
Tue Jul 9 12:06:46 MDT 2019
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi MET-Help,
I am trying to use python embedding for grib format in MET v8.
I know that MET can already handle Grib files, but python embedding would
give more flexibility.
Attached is my python script for embedding (python_script_gfs.py.txt). I am
using PYGRIB library to handle the grib format.
When I run this script just as a python, it basically works. It extracts a
given variables into NUMPY array and give some plot.
When I use this as python embedding in MET, it gives "Segmentation fault
(core dumped)" at Line 47. This line extracts the data values of given Grib
Record from Grib file, and returns as a NUMPY array. I have used very basic
MET executable to test this.
$ plot_data_plane PYTHON_NUMPY test.ps 'name="python_script_gfs.py";'
I am testing this issue on NCAR Cheyenne machine with following environment.
$ module purge
$ module use /glade/p/ral/jntp/MET/MET_releases/modulefiles
$ module load met/8.0
$ ncar_pylib
FYI, I can successfully use python embedding of NetCDF files for
plot_data_plane or grid_stat.
Thank you for your help!
BJ
--
Byoung-Joo (BJ) Jung
National Center for Atmospheric Research
Mesoscale and Microscale Meteorology Laboratory
PO Box 3000, Boulder CO 80307
Phone: 303-497-8151
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Python embedding for grib format
From: John Halley Gotway
Time: Mon Apr 15 13:04:20 2019
Hi BJ,
I see you're having trouble running your python script through MET on
cheyenne. I tried it myself and see the same behavior you describe.
I'm
going to assign this ticket to Julie Prestopnik, the person who
compiles
MET on many platforms, including cheyenne. I'm wondering how she
configured MET on cheyenne.
I suspect it was not using the ncar_pylib environment. Perhaps, she
could
try recompiling MET to be consistent with that environment to see if
that
fixes this issue?
Thanks,
John
On Fri, Apr 12, 2019 at 11:42 AM bjung at ucar.edu via RT
<met_help at ucar.edu>
wrote:
>
> Fri Apr 12 11:37:05 2019: Request 89704 was acted upon.
> Transaction: Ticket created by bjung at ucar.edu
> Queue: met_help
> Subject: Python embedding for grib format
> Owner: Nobody
> Requestors: bjung at ucar.edu
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704 >
>
>
> Hi MET-Help,
>
> I am trying to use python embedding for grib format in MET v8.
> I know that MET can already handle Grib files, but python embedding
would
> give more flexibility.
>
>
> Attached is my python script for embedding
(python_script_gfs.py.txt). I am
> using PYGRIB library to handle the grib format.
> When I run this script just as a python, it basically works. It
extracts a
> given variables into NUMPY array and give some plot.
> When I use this as python embedding in MET, it gives "Segmentation
fault
> (core dumped)" at Line 47. This line extracts the data values of
given Grib
> Record from Grib file, and returns as a NUMPY array. I have used
very basic
> MET executable to test this.
> $ plot_data_plane PYTHON_NUMPY test.ps
'name="python_script_gfs.py";'
>
> I am testing this issue on NCAR Cheyenne machine with following
> environment.
>
> $ module purge
>
> $ module use /glade/p/ral/jntp/MET/MET_releases/modulefiles
>
> $ module load met/8.0
>
> $ ncar_pylib
>
> FYI, I can successfully use python embedding of NetCDF files for
> plot_data_plane or grid_stat.
>
> Thank you for your help!
> BJ
>
>
>
> --
>
> Byoung-Joo (BJ) Jung
>
> National Center for Atmospheric Research
>
> Mesoscale and Microscale Meteorology Laboratory
>
> PO Box 3000, Boulder CO 80307
>
> Phone: 303-497-8151
>
>
------------------------------------------------
Subject: Python embedding for grib format
From: bjung at ucar.edu
Time: Mon Apr 15 13:20:27 2019
John,
Yes. that could be related to environmental configuration.
Thanks for having a look at this issue !
BJ
On Mon, Apr 15, 2019 at 1:04 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Hi BJ,
>
> I see you're having trouble running your python script through MET
on
> cheyenne. I tried it myself and see the same behavior you describe.
I'm
> going to assign this ticket to Julie Prestopnik, the person who
compiles
> MET on many platforms, including cheyenne. I'm wondering how she
> configured MET on cheyenne.
>
> I suspect it was not using the ncar_pylib environment. Perhaps, she
could
> try recompiling MET to be consistent with that environment to see if
that
> fixes this issue?
>
> Thanks,
> John
>
> On Fri, Apr 12, 2019 at 11:42 AM bjung at ucar.edu via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Fri Apr 12 11:37:05 2019: Request 89704 was acted upon.
> > Transaction: Ticket created by bjung at ucar.edu
> > Queue: met_help
> > Subject: Python embedding for grib format
> > Owner: Nobody
> > Requestors: bjung at ucar.edu
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704 >
> >
> >
> > Hi MET-Help,
> >
> > I am trying to use python embedding for grib format in MET v8.
> > I know that MET can already handle Grib files, but python
embedding would
> > give more flexibility.
> >
> >
> > Attached is my python script for embedding
(python_script_gfs.py.txt). I
> am
> > using PYGRIB library to handle the grib format.
> > When I run this script just as a python, it basically works. It
extracts
> a
> > given variables into NUMPY array and give some plot.
> > When I use this as python embedding in MET, it gives "Segmentation
fault
> > (core dumped)" at Line 47. This line extracts the data values of
given
> Grib
> > Record from Grib file, and returns as a NUMPY array. I have used
very
> basic
> > MET executable to test this.
> > $ plot_data_plane PYTHON_NUMPY test.ps
'name="python_script_gfs.py";'
> >
> > I am testing this issue on NCAR Cheyenne machine with following
> > environment.
> >
> > $ module purge
> >
> > $ module use /glade/p/ral/jntp/MET/MET_releases/modulefiles
> >
> > $ module load met/8.0
> >
> > $ ncar_pylib
> >
> > FYI, I can successfully use python embedding of NetCDF files for
> > plot_data_plane or grid_stat.
> >
> > Thank you for your help!
> > BJ
> >
> >
> >
> > --
> >
> > Byoung-Joo (BJ) Jung
> >
> > National Center for Atmospheric Research
> >
> > Mesoscale and Microscale Meteorology Laboratory
> >
> > PO Box 3000, Boulder CO 80307
> >
> > Phone: 303-497-8151
> >
> >
>
>
--
Byoung-Joo (BJ) Jung
National Center for Atmospheric Research
Mesoscale and Microscale Meteorology Laboratory
PO Box 3000, Boulder CO 80307
Phone: 303-497-8151
------------------------------------------------
Subject: Python embedding for grib format
From: Julie Prestopnik
Time: Wed Apr 17 12:40:28 2019
Hi BJ.
I made a change to the compilation of met-8.0. I am attempting to
test
with the same environment as you, but am having difficulty. For
example,
running the commands that you described in your original email doesn't
quite work for me:
cheyenne1:jpresto> module purge
> cheyenne1:jpresto> module use
> /glade/p/ral/jntp/MET/MET_releases/modulefiles
> cheyenne1:jpresto> module load met/8.0
> cheyenne1:jpresto> ncar_pylib
> ncar_pylib: Command not found.
>
I found that I could load "pygrib" by doing the following:
> cheyenne1:jpresto> module load python/2.7.13
> cheyenne1:jpresto> module load numpy
> cheyenne1:jpresto> module load pyproj
> cheyenne1:jpresto> module load pygrib
> cheyenne1:jpresto> module use
> /glade/p/ral/jntp/MET/MET_releases/modulefiles
> cheyenne1:jpresto> module load met/8.0
> cheyenne1:jpresto> ncar_pylib
> ncar_pylib: Command not found.
>
I am not familiar with ncar_pylib, so perhaps I am just ignorant
regarding
what to do here.
Could you please retry running your script in a clean shell so that
the
changes I made will be picked up when you load met-8.0 and let me know
if
it works for you?
Thanks!
Julie
On Mon, Apr 15, 2019 at 1:20 PM bjung at ucar.edu via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704 >
>
> John,
>
> Yes. that could be related to environmental configuration.
> Thanks for having a look at this issue !
>
> BJ
>
>
> On Mon, Apr 15, 2019 at 1:04 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Hi BJ,
> >
> > I see you're having trouble running your python script through MET
on
> > cheyenne. I tried it myself and see the same behavior you
describe. I'm
> > going to assign this ticket to Julie Prestopnik, the person who
compiles
> > MET on many platforms, including cheyenne. I'm wondering how she
> > configured MET on cheyenne.
> >
> > I suspect it was not using the ncar_pylib environment. Perhaps,
she
> could
> > try recompiling MET to be consistent with that environment to see
if that
> > fixes this issue?
> >
> > Thanks,
> > John
> >
> > On Fri, Apr 12, 2019 at 11:42 AM bjung at ucar.edu via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > >
> > > Fri Apr 12 11:37:05 2019: Request 89704 was acted upon.
> > > Transaction: Ticket created by bjung at ucar.edu
> > > Queue: met_help
> > > Subject: Python embedding for grib format
> > > Owner: Nobody
> > > Requestors: bjung at ucar.edu
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704
> >
> > >
> > >
> > > Hi MET-Help,
> > >
> > > I am trying to use python embedding for grib format in MET v8.
> > > I know that MET can already handle Grib files, but python
embedding
> would
> > > give more flexibility.
> > >
> > >
> > > Attached is my python script for embedding
(python_script_gfs.py.txt).
> I
> > am
> > > using PYGRIB library to handle the grib format.
> > > When I run this script just as a python, it basically works. It
> extracts
> > a
> > > given variables into NUMPY array and give some plot.
> > > When I use this as python embedding in MET, it gives
"Segmentation
> fault
> > > (core dumped)" at Line 47. This line extracts the data values of
given
> > Grib
> > > Record from Grib file, and returns as a NUMPY array. I have used
very
> > basic
> > > MET executable to test this.
> > > $ plot_data_plane PYTHON_NUMPY test.ps
'name="python_script_gfs.py";'
> > >
> > > I am testing this issue on NCAR Cheyenne machine with following
> > > environment.
> > >
> > > $ module purge
> > >
> > > $ module use /glade/p/ral/jntp/MET/MET_releases/modulefiles
> > >
> > > $ module load met/8.0
> > >
> > > $ ncar_pylib
> > >
> > > FYI, I can successfully use python embedding of NetCDF files for
> > > plot_data_plane or grid_stat.
> > >
> > > Thank you for your help!
> > > BJ
> > >
> > >
> > >
> > > --
> > >
> > > Byoung-Joo (BJ) Jung
> > >
> > > National Center for Atmospheric Research
> > >
> > > Mesoscale and Microscale Meteorology Laboratory
> > >
> > > PO Box 3000, Boulder CO 80307
> > >
> > > Phone: 303-497-8151
> > >
> > >
> >
> >
>
> --
>
> Byoung-Joo (BJ) Jung
>
> National Center for Atmospheric Research
>
> Mesoscale and Microscale Meteorology Laboratory
>
> PO Box 3000, Boulder CO 80307
>
> Phone: 303-497-8151
>
>
------------------------------------------------
Subject: Python embedding for grib format
From: bjung at ucar.edu
Time: Wed Apr 17 14:01:47 2019
Hi Julie,
Thanks for having a look at this issue.
Just tried it, but it also fails for me to run "ncar_pylib" with above
configuration.
I will figure out what happened , and whether there is other way to
load
Python Libraries properly later today.
Thanks,
BJ
On Wed, Apr 17, 2019 at 12:41 PM Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:
> Hi BJ.
>
> I made a change to the compilation of met-8.0. I am attempting to
test
> with the same environment as you, but am having difficulty. For
example,
> running the commands that you described in your original email
doesn't
> quite work for me:
>
> cheyenne1:jpresto> module purge
> > cheyenne1:jpresto> module use
> > /glade/p/ral/jntp/MET/MET_releases/modulefiles
> > cheyenne1:jpresto> module load met/8.0
> > cheyenne1:jpresto> ncar_pylib
> > ncar_pylib: Command not found.
> >
>
> I found that I could load "pygrib" by doing the following:
>
> > cheyenne1:jpresto> module load python/2.7.13
> > cheyenne1:jpresto> module load numpy
> > cheyenne1:jpresto> module load pyproj
> > cheyenne1:jpresto> module load pygrib
> > cheyenne1:jpresto> module use
> > /glade/p/ral/jntp/MET/MET_releases/modulefiles
> > cheyenne1:jpresto> module load met/8.0
> > cheyenne1:jpresto> ncar_pylib
> > ncar_pylib: Command not found.
> >
>
> I am not familiar with ncar_pylib, so perhaps I am just ignorant
regarding
> what to do here.
>
> Could you please retry running your script in a clean shell so that
the
> changes I made will be picked up when you load met-8.0 and let me
know if
> it works for you?
>
> Thanks!
>
> Julie
>
>
>
> On Mon, Apr 15, 2019 at 1:20 PM bjung at ucar.edu via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704 >
> >
> > John,
> >
> > Yes. that could be related to environmental configuration.
> > Thanks for having a look at this issue !
> >
> > BJ
> >
> >
> > On Mon, Apr 15, 2019 at 1:04 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Hi BJ,
> > >
> > > I see you're having trouble running your python script through
MET on
> > > cheyenne. I tried it myself and see the same behavior you
describe.
> I'm
> > > going to assign this ticket to Julie Prestopnik, the person who
> compiles
> > > MET on many platforms, including cheyenne. I'm wondering how
she
> > > configured MET on cheyenne.
> > >
> > > I suspect it was not using the ncar_pylib environment. Perhaps,
she
> > could
> > > try recompiling MET to be consistent with that environment to
see if
> that
> > > fixes this issue?
> > >
> > > Thanks,
> > > John
> > >
> > > On Fri, Apr 12, 2019 at 11:42 AM bjung at ucar.edu via RT <
> > met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > Fri Apr 12 11:37:05 2019: Request 89704 was acted upon.
> > > > Transaction: Ticket created by bjung at ucar.edu
> > > > Queue: met_help
> > > > Subject: Python embedding for grib format
> > > > Owner: Nobody
> > > > Requestors: bjung at ucar.edu
> > > > Status: new
> > > > Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704
> > >
> > > >
> > > >
> > > > Hi MET-Help,
> > > >
> > > > I am trying to use python embedding for grib format in MET v8.
> > > > I know that MET can already handle Grib files, but python
embedding
> > would
> > > > give more flexibility.
> > > >
> > > >
> > > > Attached is my python script for embedding
> (python_script_gfs.py.txt).
> > I
> > > am
> > > > using PYGRIB library to handle the grib format.
> > > > When I run this script just as a python, it basically works.
It
> > extracts
> > > a
> > > > given variables into NUMPY array and give some plot.
> > > > When I use this as python embedding in MET, it gives
"Segmentation
> > fault
> > > > (core dumped)" at Line 47. This line extracts the data values
of
> given
> > > Grib
> > > > Record from Grib file, and returns as a NUMPY array. I have
used very
> > > basic
> > > > MET executable to test this.
> > > > $ plot_data_plane PYTHON_NUMPY test.ps
> 'name="python_script_gfs.py";'
> > > >
> > > > I am testing this issue on NCAR Cheyenne machine with
following
> > > > environment.
> > > >
> > > > $ module purge
> > > >
> > > > $ module use /glade/p/ral/jntp/MET/MET_releases/modulefiles
> > > >
> > > > $ module load met/8.0
> > > >
> > > > $ ncar_pylib
> > > >
> > > > FYI, I can successfully use python embedding of NetCDF files
for
> > > > plot_data_plane or grid_stat.
> > > >
> > > > Thank you for your help!
> > > > BJ
> > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Byoung-Joo (BJ) Jung
> > > >
> > > > National Center for Atmospheric Research
> > > >
> > > > Mesoscale and Microscale Meteorology Laboratory
> > > >
> > > > PO Box 3000, Boulder CO 80307
> > > >
> > > > Phone: 303-497-8151
> > > >
> > > >
> > >
> > >
> >
> > --
> >
> > Byoung-Joo (BJ) Jung
> >
> > National Center for Atmospheric Research
> >
> > Mesoscale and Microscale Meteorology Laboratory
> >
> > PO Box 3000, Boulder CO 80307
> >
> > Phone: 303-497-8151
> >
> >
>
>
--
Byoung-Joo (BJ) Jung
National Center for Atmospheric Research
Mesoscale and Microscale Meteorology Laboratory
PO Box 3000, Boulder CO 80307
Phone: 303-497-8151
------------------------------------------------
Subject: Python embedding for grib format
From: Julie Prestopnik
Time: Wed Apr 17 14:46:02 2019
Thank you. I'll wait to hear back from you.
Julie
On Wed, Apr 17, 2019 at 2:01 PM bjung at ucar.edu via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704 >
>
> Hi Julie,
> Thanks for having a look at this issue.
>
> Just tried it, but it also fails for me to run "ncar_pylib" with
above
> configuration.
> I will figure out what happened , and whether there is other way to
load
> Python Libraries properly later today.
>
> Thanks,
> BJ
>
> On Wed, Apr 17, 2019 at 12:41 PM Julie Prestopnik via RT <
> met_help at ucar.edu>
> wrote:
>
> > Hi BJ.
> >
> > I made a change to the compilation of met-8.0. I am attempting to
test
> > with the same environment as you, but am having difficulty. For
example,
> > running the commands that you described in your original email
doesn't
> > quite work for me:
> >
> > cheyenne1:jpresto> module purge
> > > cheyenne1:jpresto> module use
> > > /glade/p/ral/jntp/MET/MET_releases/modulefiles
> > > cheyenne1:jpresto> module load met/8.0
> > > cheyenne1:jpresto> ncar_pylib
> > > ncar_pylib: Command not found.
> > >
> >
> > I found that I could load "pygrib" by doing the following:
> >
> > > cheyenne1:jpresto> module load python/2.7.13
> > > cheyenne1:jpresto> module load numpy
> > > cheyenne1:jpresto> module load pyproj
> > > cheyenne1:jpresto> module load pygrib
> > > cheyenne1:jpresto> module use
> > > /glade/p/ral/jntp/MET/MET_releases/modulefiles
> > > cheyenne1:jpresto> module load met/8.0
> > > cheyenne1:jpresto> ncar_pylib
> > > ncar_pylib: Command not found.
> > >
> >
> > I am not familiar with ncar_pylib, so perhaps I am just ignorant
> regarding
> > what to do here.
> >
> > Could you please retry running your script in a clean shell so
that the
> > changes I made will be picked up when you load met-8.0 and let me
know if
> > it works for you?
> >
> > Thanks!
> >
> > Julie
> >
> >
> >
> > On Mon, Apr 15, 2019 at 1:20 PM bjung at ucar.edu via RT
<met_help at ucar.edu
> >
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704 >
> > >
> > > John,
> > >
> > > Yes. that could be related to environmental configuration.
> > > Thanks for having a look at this issue !
> > >
> > > BJ
> > >
> > >
> > > On Mon, Apr 15, 2019 at 1:04 PM John Halley Gotway via RT <
> > > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Hi BJ,
> > > >
> > > > I see you're having trouble running your python script through
MET on
> > > > cheyenne. I tried it myself and see the same behavior you
describe.
> > I'm
> > > > going to assign this ticket to Julie Prestopnik, the person
who
> > compiles
> > > > MET on many platforms, including cheyenne. I'm wondering how
she
> > > > configured MET on cheyenne.
> > > >
> > > > I suspect it was not using the ncar_pylib environment.
Perhaps, she
> > > could
> > > > try recompiling MET to be consistent with that environment to
see if
> > that
> > > > fixes this issue?
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Fri, Apr 12, 2019 at 11:42 AM bjung at ucar.edu via RT <
> > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > Fri Apr 12 11:37:05 2019: Request 89704 was acted upon.
> > > > > Transaction: Ticket created by bjung at ucar.edu
> > > > > Queue: met_help
> > > > > Subject: Python embedding for grib format
> > > > > Owner: Nobody
> > > > > Requestors: bjung at ucar.edu
> > > > > Status: new
> > > > > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704
> > > >
> > > > >
> > > > >
> > > > > Hi MET-Help,
> > > > >
> > > > > I am trying to use python embedding for grib format in MET
v8.
> > > > > I know that MET can already handle Grib files, but python
embedding
> > > would
> > > > > give more flexibility.
> > > > >
> > > > >
> > > > > Attached is my python script for embedding
> > (python_script_gfs.py.txt).
> > > I
> > > > am
> > > > > using PYGRIB library to handle the grib format.
> > > > > When I run this script just as a python, it basically works.
It
> > > extracts
> > > > a
> > > > > given variables into NUMPY array and give some plot.
> > > > > When I use this as python embedding in MET, it gives
"Segmentation
> > > fault
> > > > > (core dumped)" at Line 47. This line extracts the data
values of
> > given
> > > > Grib
> > > > > Record from Grib file, and returns as a NUMPY array. I have
used
> very
> > > > basic
> > > > > MET executable to test this.
> > > > > $ plot_data_plane PYTHON_NUMPY test.ps
> > 'name="python_script_gfs.py";'
> > > > >
> > > > > I am testing this issue on NCAR Cheyenne machine with
following
> > > > > environment.
> > > > >
> > > > > $ module purge
> > > > >
> > > > > $ module use /glade/p/ral/jntp/MET/MET_releases/modulefiles
> > > > >
> > > > > $ module load met/8.0
> > > > >
> > > > > $ ncar_pylib
> > > > >
> > > > > FYI, I can successfully use python embedding of NetCDF files
for
> > > > > plot_data_plane or grid_stat.
> > > > >
> > > > > Thank you for your help!
> > > > > BJ
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Byoung-Joo (BJ) Jung
> > > > >
> > > > > National Center for Atmospheric Research
> > > > >
> > > > > Mesoscale and Microscale Meteorology Laboratory
> > > > >
> > > > > PO Box 3000, Boulder CO 80307
> > > > >
> > > > > Phone: 303-497-8151
> > > > >
> > > > >
> > > >
> > > >
> > >
> > > --
> > >
> > > Byoung-Joo (BJ) Jung
> > >
> > > National Center for Atmospheric Research
> > >
> > > Mesoscale and Microscale Meteorology Laboratory
> > >
> > > PO Box 3000, Boulder CO 80307
> > >
> > > Phone: 303-497-8151
> > >
> > >
> >
> >
>
> --
>
> Byoung-Joo (BJ) Jung
>
> National Center for Atmospheric Research
>
> Mesoscale and Microscale Meteorology Laboratory
>
> PO Box 3000, Boulder CO 80307
>
> Phone: 303-497-8151
>
>
------------------------------------------------
Subject: Python embedding for grib format
From: bjung at ucar.edu
Time: Thu Apr 18 13:21:56 2019
Hi Julie,
First, "ncar_pylib" is an environment to load "useful" python
libraries.
You can see the information here:
https://www2.cisl.ucar.edu/resources/computational-
systems/cheyenne/software/python
I have ask CISL whether they have changed or removed "ncar_pylib"
capabilities with Python/2.7.13, and they replied as below.
> Thanks for the email. The "ncar_pylib" virtual environments have
never
been available in Python
> 2.7.13. We introduced that capability with 2.7.14, and it is
available in
2.7.15 as well. It's
> possible that you were activating the virtual environments in some
other
way, but the only
> supported versions are 2.7.14, 2.7.15, 3.6.4, and 3.6.8 (the current
default). I would recommend
> that you switch to one of those versions of Python to use the NPL.
I can not remember which python version was used to build the "pre-
built"
of met v8 on Cheyenne (Which was working until April 16th).
But *currently*, that seems to be *python/2.7.13* . The met modules &
binary files are refreshed on April 17.
Also, basic python embedding does not work with these new binary
files.
Can we try to *build* met v8 with *python/2.7.14* and try to reproduce
my
original error ?
Thanks,
BJ
cc: Craig Schwartz
On Wed, Apr 17, 2019 at 2:46 PM Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:
> Thank you. I'll wait to hear back from you.
>
> Julie
>
> On Wed, Apr 17, 2019 at 2:01 PM bjung at ucar.edu via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704 >
> >
> > Hi Julie,
> > Thanks for having a look at this issue.
> >
> > Just tried it, but it also fails for me to run "ncar_pylib" with
above
> > configuration.
> > I will figure out what happened , and whether there is other way
to load
> > Python Libraries properly later today.
> >
> > Thanks,
> > BJ
> >
> > On Wed, Apr 17, 2019 at 12:41 PM Julie Prestopnik via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Hi BJ.
> > >
> > > I made a change to the compilation of met-8.0. I am attempting
to test
> > > with the same environment as you, but am having difficulty. For
> example,
> > > running the commands that you described in your original email
doesn't
> > > quite work for me:
> > >
> > > cheyenne1:jpresto> module purge
> > > > cheyenne1:jpresto> module use
> > > > /glade/p/ral/jntp/MET/MET_releases/modulefiles
> > > > cheyenne1:jpresto> module load met/8.0
> > > > cheyenne1:jpresto> ncar_pylib
> > > > ncar_pylib: Command not found.
> > > >
> > >
> > > I found that I could load "pygrib" by doing the following:
> > >
> > > > cheyenne1:jpresto> module load python/2.7.13
> > > > cheyenne1:jpresto> module load numpy
> > > > cheyenne1:jpresto> module load pyproj
> > > > cheyenne1:jpresto> module load pygrib
> > > > cheyenne1:jpresto> module use
> > > > /glade/p/ral/jntp/MET/MET_releases/modulefiles
> > > > cheyenne1:jpresto> module load met/8.0
> > > > cheyenne1:jpresto> ncar_pylib
> > > > ncar_pylib: Command not found.
> > > >
> > >
> > > I am not familiar with ncar_pylib, so perhaps I am just ignorant
> > regarding
> > > what to do here.
> > >
> > > Could you please retry running your script in a clean shell so
that the
> > > changes I made will be picked up when you load met-8.0 and let
me know
> if
> > > it works for you?
> > >
> > > Thanks!
> > >
> > > Julie
> > >
> > >
> > >
> > > On Mon, Apr 15, 2019 at 1:20 PM bjung at ucar.edu via RT <
> met_help at ucar.edu
> > >
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704
>
> > > >
> > > > John,
> > > >
> > > > Yes. that could be related to environmental configuration.
> > > > Thanks for having a look at this issue !
> > > >
> > > > BJ
> > > >
> > > >
> > > > On Mon, Apr 15, 2019 at 1:04 PM John Halley Gotway via RT <
> > > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Hi BJ,
> > > > >
> > > > > I see you're having trouble running your python script
through MET
> on
> > > > > cheyenne. I tried it myself and see the same behavior you
> describe.
> > > I'm
> > > > > going to assign this ticket to Julie Prestopnik, the person
who
> > > compiles
> > > > > MET on many platforms, including cheyenne. I'm wondering
how she
> > > > > configured MET on cheyenne.
> > > > >
> > > > > I suspect it was not using the ncar_pylib environment.
Perhaps,
> she
> > > > could
> > > > > try recompiling MET to be consistent with that environment
to see
> if
> > > that
> > > > > fixes this issue?
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > > On Fri, Apr 12, 2019 at 11:42 AM bjung at ucar.edu via RT <
> > > > met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > Fri Apr 12 11:37:05 2019: Request 89704 was acted upon.
> > > > > > Transaction: Ticket created by bjung at ucar.edu
> > > > > > Queue: met_help
> > > > > > Subject: Python embedding for grib format
> > > > > > Owner: Nobody
> > > > > > Requestors: bjung at ucar.edu
> > > > > > Status: new
> > > > > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704
> > > > >
> > > > > >
> > > > > >
> > > > > > Hi MET-Help,
> > > > > >
> > > > > > I am trying to use python embedding for grib format in MET
v8.
> > > > > > I know that MET can already handle Grib files, but python
> embedding
> > > > would
> > > > > > give more flexibility.
> > > > > >
> > > > > >
> > > > > > Attached is my python script for embedding
> > > (python_script_gfs.py.txt).
> > > > I
> > > > > am
> > > > > > using PYGRIB library to handle the grib format.
> > > > > > When I run this script just as a python, it basically
works. It
> > > > extracts
> > > > > a
> > > > > > given variables into NUMPY array and give some plot.
> > > > > > When I use this as python embedding in MET, it gives
> "Segmentation
> > > > fault
> > > > > > (core dumped)" at Line 47. This line extracts the data
values of
> > > given
> > > > > Grib
> > > > > > Record from Grib file, and returns as a NUMPY array. I
have used
> > very
> > > > > basic
> > > > > > MET executable to test this.
> > > > > > $ plot_data_plane PYTHON_NUMPY test.ps
> > > 'name="python_script_gfs.py";'
> > > > > >
> > > > > > I am testing this issue on NCAR Cheyenne machine with
following
> > > > > > environment.
> > > > > >
> > > > > > $ module purge
> > > > > >
> > > > > > $ module use
/glade/p/ral/jntp/MET/MET_releases/modulefiles
> > > > > >
> > > > > > $ module load met/8.0
> > > > > >
> > > > > > $ ncar_pylib
> > > > > >
> > > > > > FYI, I can successfully use python embedding of NetCDF
files for
> > > > > > plot_data_plane or grid_stat.
> > > > > >
> > > > > > Thank you for your help!
> > > > > > BJ
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Byoung-Joo (BJ) Jung
> > > > > >
> > > > > > National Center for Atmospheric Research
> > > > > >
> > > > > > Mesoscale and Microscale Meteorology Laboratory
> > > > > >
> > > > > > PO Box 3000, Boulder CO 80307
> > > > > >
> > > > > > Phone: 303-497-8151
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > > --
> > > >
> > > > Byoung-Joo (BJ) Jung
> > > >
> > > > National Center for Atmospheric Research
> > > >
> > > > Mesoscale and Microscale Meteorology Laboratory
> > > >
> > > > PO Box 3000, Boulder CO 80307
> > > >
> > > > Phone: 303-497-8151
> > > >
> > > >
> > >
> > >
> >
> > --
> >
> > Byoung-Joo (BJ) Jung
> >
> > National Center for Atmospheric Research
> >
> > Mesoscale and Microscale Meteorology Laboratory
> >
> > PO Box 3000, Boulder CO 80307
> >
> > Phone: 303-497-8151
> >
> >
>
>
--
Byoung-Joo (BJ) Jung
National Center for Atmospheric Research
Mesoscale and Microscale Meteorology Laboratory
PO Box 3000, Boulder CO 80307
Phone: 303-497-8151
------------------------------------------------
Subject: Python embedding for grib format
From: Julie Prestopnik
Time: Thu Apr 18 15:45:28 2019
>
> Can we try to *build* met v8 with *python/2.7.14* and try to
reproduce my
> original error ?
Yes. When I loaded pygrib, I saw Python 2.7.13 being used, so I
rebuilt
met-8.0 using Python 2.7.13 (it had been using 2.7.14). I will
rebuild
using 2.7.14 and we'll see if we can reproduce your original error. I
may
not be able do to this until next week however as I will be out of the
office most of tomorrow.
I will follow up once I have reinstalled met-8.0
Julie
On Thu, Apr 18, 2019 at 1:21 PM bjung at ucar.edu via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704 >
>
> Hi Julie,
>
> First, "ncar_pylib" is an environment to load "useful" python
libraries.
> You can see the information here:
>
> https://www2.cisl.ucar.edu/resources/computational-
systems/cheyenne/software/python
>
> I have ask CISL whether they have changed or removed "ncar_pylib"
> capabilities with Python/2.7.13, and they replied as below.
>
> > Thanks for the email. The "ncar_pylib" virtual environments have
never
> been available in Python
> > 2.7.13. We introduced that capability with 2.7.14, and it is
available in
> 2.7.15 as well. It's
> > possible that you were activating the virtual environments in some
other
> way, but the only
> > supported versions are 2.7.14, 2.7.15, 3.6.4, and 3.6.8 (the
current
> default). I would recommend
> > that you switch to one of those versions of Python to use the NPL.
>
>
> I can not remember which python version was used to build the "pre-
built"
> of met v8 on Cheyenne (Which was working until April 16th).
> But *currently*, that seems to be *python/2.7.13* . The met modules
&
> binary files are refreshed on April 17.
> Also, basic python embedding does not work with these new binary
files.
>
> Can we try to *build* met v8 with *python/2.7.14* and try to
reproduce my
> original error ?
>
> Thanks,
> BJ
>
> cc: Craig Schwartz
>
>
>
> On Wed, Apr 17, 2019 at 2:46 PM Julie Prestopnik via RT
<met_help at ucar.edu
> >
> wrote:
>
> > Thank you. I'll wait to hear back from you.
> >
> > Julie
> >
> > On Wed, Apr 17, 2019 at 2:01 PM bjung at ucar.edu via RT
<met_help at ucar.edu
> >
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704 >
> > >
> > > Hi Julie,
> > > Thanks for having a look at this issue.
> > >
> > > Just tried it, but it also fails for me to run "ncar_pylib" with
above
> > > configuration.
> > > I will figure out what happened , and whether there is other way
to
> load
> > > Python Libraries properly later today.
> > >
> > > Thanks,
> > > BJ
> > >
> > > On Wed, Apr 17, 2019 at 12:41 PM Julie Prestopnik via RT <
> > > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Hi BJ.
> > > >
> > > > I made a change to the compilation of met-8.0. I am
attempting to
> test
> > > > with the same environment as you, but am having difficulty.
For
> > example,
> > > > running the commands that you described in your original email
> doesn't
> > > > quite work for me:
> > > >
> > > > cheyenne1:jpresto> module purge
> > > > > cheyenne1:jpresto> module use
> > > > > /glade/p/ral/jntp/MET/MET_releases/modulefiles
> > > > > cheyenne1:jpresto> module load met/8.0
> > > > > cheyenne1:jpresto> ncar_pylib
> > > > > ncar_pylib: Command not found.
> > > > >
> > > >
> > > > I found that I could load "pygrib" by doing the following:
> > > >
> > > > > cheyenne1:jpresto> module load python/2.7.13
> > > > > cheyenne1:jpresto> module load numpy
> > > > > cheyenne1:jpresto> module load pyproj
> > > > > cheyenne1:jpresto> module load pygrib
> > > > > cheyenne1:jpresto> module use
> > > > > /glade/p/ral/jntp/MET/MET_releases/modulefiles
> > > > > cheyenne1:jpresto> module load met/8.0
> > > > > cheyenne1:jpresto> ncar_pylib
> > > > > ncar_pylib: Command not found.
> > > > >
> > > >
> > > > I am not familiar with ncar_pylib, so perhaps I am just
ignorant
> > > regarding
> > > > what to do here.
> > > >
> > > > Could you please retry running your script in a clean shell so
that
> the
> > > > changes I made will be picked up when you load met-8.0 and let
me
> know
> > if
> > > > it works for you?
> > > >
> > > > Thanks!
> > > >
> > > > Julie
> > > >
> > > >
> > > >
> > > > On Mon, Apr 15, 2019 at 1:20 PM bjung at ucar.edu via RT <
> > met_help at ucar.edu
> > > >
> > > > wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704 >
> > > > >
> > > > > John,
> > > > >
> > > > > Yes. that could be related to environmental configuration.
> > > > > Thanks for having a look at this issue !
> > > > >
> > > > > BJ
> > > > >
> > > > >
> > > > > On Mon, Apr 15, 2019 at 1:04 PM John Halley Gotway via RT <
> > > > > met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > > Hi BJ,
> > > > > >
> > > > > > I see you're having trouble running your python script
through
> MET
> > on
> > > > > > cheyenne. I tried it myself and see the same behavior you
> > describe.
> > > > I'm
> > > > > > going to assign this ticket to Julie Prestopnik, the
person who
> > > > compiles
> > > > > > MET on many platforms, including cheyenne. I'm wondering
how she
> > > > > > configured MET on cheyenne.
> > > > > >
> > > > > > I suspect it was not using the ncar_pylib environment.
Perhaps,
> > she
> > > > > could
> > > > > > try recompiling MET to be consistent with that environment
to see
> > if
> > > > that
> > > > > > fixes this issue?
> > > > > >
> > > > > > Thanks,
> > > > > > John
> > > > > >
> > > > > > On Fri, Apr 12, 2019 at 11:42 AM bjung at ucar.edu via RT <
> > > > > met_help at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > > >
> > > > > > > Fri Apr 12 11:37:05 2019: Request 89704 was acted upon.
> > > > > > > Transaction: Ticket created by bjung at ucar.edu
> > > > > > > Queue: met_help
> > > > > > > Subject: Python embedding for grib format
> > > > > > > Owner: Nobody
> > > > > > > Requestors: bjung at ucar.edu
> > > > > > > Status: new
> > > > > > > Ticket <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Hi MET-Help,
> > > > > > >
> > > > > > > I am trying to use python embedding for grib format in
MET v8.
> > > > > > > I know that MET can already handle Grib files, but
python
> > embedding
> > > > > would
> > > > > > > give more flexibility.
> > > > > > >
> > > > > > >
> > > > > > > Attached is my python script for embedding
> > > > (python_script_gfs.py.txt).
> > > > > I
> > > > > > am
> > > > > > > using PYGRIB library to handle the grib format.
> > > > > > > When I run this script just as a python, it basically
works. It
> > > > > extracts
> > > > > > a
> > > > > > > given variables into NUMPY array and give some plot.
> > > > > > > When I use this as python embedding in MET, it gives
> > "Segmentation
> > > > > fault
> > > > > > > (core dumped)" at Line 47. This line extracts the data
values
> of
> > > > given
> > > > > > Grib
> > > > > > > Record from Grib file, and returns as a NUMPY array. I
have
> used
> > > very
> > > > > > basic
> > > > > > > MET executable to test this.
> > > > > > > $ plot_data_plane PYTHON_NUMPY test.ps
> > > > 'name="python_script_gfs.py";'
> > > > > > >
> > > > > > > I am testing this issue on NCAR Cheyenne machine with
following
> > > > > > > environment.
> > > > > > >
> > > > > > > $ module purge
> > > > > > >
> > > > > > > $ module use
/glade/p/ral/jntp/MET/MET_releases/modulefiles
> > > > > > >
> > > > > > > $ module load met/8.0
> > > > > > >
> > > > > > > $ ncar_pylib
> > > > > > >
> > > > > > > FYI, I can successfully use python embedding of NetCDF
files
> for
> > > > > > > plot_data_plane or grid_stat.
> > > > > > >
> > > > > > > Thank you for your help!
> > > > > > > BJ
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > >
> > > > > > > Byoung-Joo (BJ) Jung
> > > > > > >
> > > > > > > National Center for Atmospheric Research
> > > > > > >
> > > > > > > Mesoscale and Microscale Meteorology Laboratory
> > > > > > >
> > > > > > > PO Box 3000, Boulder CO 80307
> > > > > > >
> > > > > > > Phone: 303-497-8151
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Byoung-Joo (BJ) Jung
> > > > >
> > > > > National Center for Atmospheric Research
> > > > >
> > > > > Mesoscale and Microscale Meteorology Laboratory
> > > > >
> > > > > PO Box 3000, Boulder CO 80307
> > > > >
> > > > > Phone: 303-497-8151
> > > > >
> > > > >
> > > >
> > > >
> > >
> > > --
> > >
> > > Byoung-Joo (BJ) Jung
> > >
> > > National Center for Atmospheric Research
> > >
> > > Mesoscale and Microscale Meteorology Laboratory
> > >
> > > PO Box 3000, Boulder CO 80307
> > >
> > > Phone: 303-497-8151
> > >
> > >
> >
> >
>
> --
>
> Byoung-Joo (BJ) Jung
>
> National Center for Atmospheric Research
>
> Mesoscale and Microscale Meteorology Laboratory
>
> PO Box 3000, Boulder CO 80307
>
> Phone: 303-497-8151
>
>
------------------------------------------------
Subject: Python embedding for grib format
From: bjung at ucar.edu
Time: Thu Apr 18 16:46:20 2019
Thanks for your efforts, Julie.
BJ
On Thu, Apr 18, 2019 at 3:45 PM Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:
> >
> > Can we try to *build* met v8 with *python/2.7.14* and try to
reproduce my
> > original error ?
>
>
> Yes. When I loaded pygrib, I saw Python 2.7.13 being used, so I
rebuilt
> met-8.0 using Python 2.7.13 (it had been using 2.7.14). I will
rebuild
> using 2.7.14 and we'll see if we can reproduce your original error.
I may
> not be able do to this until next week however as I will be out of
the
> office most of tomorrow.
>
> I will follow up once I have reinstalled met-8.0
>
> Julie
>
> On Thu, Apr 18, 2019 at 1:21 PM bjung at ucar.edu via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704 >
> >
> > Hi Julie,
> >
> > First, "ncar_pylib" is an environment to load "useful" python
libraries.
> > You can see the information here:
> >
> >
> https://www2.cisl.ucar.edu/resources/computational-
systems/cheyenne/software/python
> >
> > I have ask CISL whether they have changed or removed "ncar_pylib"
> > capabilities with Python/2.7.13, and they replied as below.
> >
> > > Thanks for the email. The "ncar_pylib" virtual environments have
never
> > been available in Python
> > > 2.7.13. We introduced that capability with 2.7.14, and it is
available
> in
> > 2.7.15 as well. It's
> > > possible that you were activating the virtual environments in
some
> other
> > way, but the only
> > > supported versions are 2.7.14, 2.7.15, 3.6.4, and 3.6.8 (the
current
> > default). I would recommend
> > > that you switch to one of those versions of Python to use the
NPL.
> >
> >
> > I can not remember which python version was used to build the
"pre-built"
> > of met v8 on Cheyenne (Which was working until April 16th).
> > But *currently*, that seems to be *python/2.7.13* . The met
modules &
> > binary files are refreshed on April 17.
> > Also, basic python embedding does not work with these new binary
files.
> >
> > Can we try to *build* met v8 with *python/2.7.14* and try to
reproduce my
> > original error ?
> >
> > Thanks,
> > BJ
> >
> > cc: Craig Schwartz
> >
> >
> >
> > On Wed, Apr 17, 2019 at 2:46 PM Julie Prestopnik via RT <
> met_help at ucar.edu
> > >
> > wrote:
> >
> > > Thank you. I'll wait to hear back from you.
> > >
> > > Julie
> > >
> > > On Wed, Apr 17, 2019 at 2:01 PM bjung at ucar.edu via RT <
> met_help at ucar.edu
> > >
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704
>
> > > >
> > > > Hi Julie,
> > > > Thanks for having a look at this issue.
> > > >
> > > > Just tried it, but it also fails for me to run "ncar_pylib"
with
> above
> > > > configuration.
> > > > I will figure out what happened , and whether there is other
way to
> > load
> > > > Python Libraries properly later today.
> > > >
> > > > Thanks,
> > > > BJ
> > > >
> > > > On Wed, Apr 17, 2019 at 12:41 PM Julie Prestopnik via RT <
> > > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Hi BJ.
> > > > >
> > > > > I made a change to the compilation of met-8.0. I am
attempting to
> > test
> > > > > with the same environment as you, but am having difficulty.
For
> > > example,
> > > > > running the commands that you described in your original
email
> > doesn't
> > > > > quite work for me:
> > > > >
> > > > > cheyenne1:jpresto> module purge
> > > > > > cheyenne1:jpresto> module use
> > > > > > /glade/p/ral/jntp/MET/MET_releases/modulefiles
> > > > > > cheyenne1:jpresto> module load met/8.0
> > > > > > cheyenne1:jpresto> ncar_pylib
> > > > > > ncar_pylib: Command not found.
> > > > > >
> > > > >
> > > > > I found that I could load "pygrib" by doing the following:
> > > > >
> > > > > > cheyenne1:jpresto> module load python/2.7.13
> > > > > > cheyenne1:jpresto> module load numpy
> > > > > > cheyenne1:jpresto> module load pyproj
> > > > > > cheyenne1:jpresto> module load pygrib
> > > > > > cheyenne1:jpresto> module use
> > > > > > /glade/p/ral/jntp/MET/MET_releases/modulefiles
> > > > > > cheyenne1:jpresto> module load met/8.0
> > > > > > cheyenne1:jpresto> ncar_pylib
> > > > > > ncar_pylib: Command not found.
> > > > > >
> > > > >
> > > > > I am not familiar with ncar_pylib, so perhaps I am just
ignorant
> > > > regarding
> > > > > what to do here.
> > > > >
> > > > > Could you please retry running your script in a clean shell
so that
> > the
> > > > > changes I made will be picked up when you load met-8.0 and
let me
> > know
> > > if
> > > > > it works for you?
> > > > >
> > > > > Thanks!
> > > > >
> > > > > Julie
> > > > >
> > > > >
> > > > >
> > > > > On Mon, Apr 15, 2019 at 1:20 PM bjung at ucar.edu via RT <
> > > met_help at ucar.edu
> > > > >
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704 >
> > > > > >
> > > > > > John,
> > > > > >
> > > > > > Yes. that could be related to environmental configuration.
> > > > > > Thanks for having a look at this issue !
> > > > > >
> > > > > > BJ
> > > > > >
> > > > > >
> > > > > > On Mon, Apr 15, 2019 at 1:04 PM John Halley Gotway via RT
<
> > > > > > met_help at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi BJ,
> > > > > > >
> > > > > > > I see you're having trouble running your python script
through
> > MET
> > > on
> > > > > > > cheyenne. I tried it myself and see the same behavior
you
> > > describe.
> > > > > I'm
> > > > > > > going to assign this ticket to Julie Prestopnik, the
person who
> > > > > compiles
> > > > > > > MET on many platforms, including cheyenne. I'm
wondering how
> she
> > > > > > > configured MET on cheyenne.
> > > > > > >
> > > > > > > I suspect it was not using the ncar_pylib environment.
> Perhaps,
> > > she
> > > > > > could
> > > > > > > try recompiling MET to be consistent with that
environment to
> see
> > > if
> > > > > that
> > > > > > > fixes this issue?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > John
> > > > > > >
> > > > > > > On Fri, Apr 12, 2019 at 11:42 AM bjung at ucar.edu via RT <
> > > > > > met_help at ucar.edu>
> > > > > > > wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > Fri Apr 12 11:37:05 2019: Request 89704 was acted
upon.
> > > > > > > > Transaction: Ticket created by bjung at ucar.edu
> > > > > > > > Queue: met_help
> > > > > > > > Subject: Python embedding for grib format
> > > > > > > > Owner: Nobody
> > > > > > > > Requestors: bjung at ucar.edu
> > > > > > > > Status: new
> > > > > > > > Ticket <URL:
> > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704
> > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Hi MET-Help,
> > > > > > > >
> > > > > > > > I am trying to use python embedding for grib format in
MET
> v8.
> > > > > > > > I know that MET can already handle Grib files, but
python
> > > embedding
> > > > > > would
> > > > > > > > give more flexibility.
> > > > > > > >
> > > > > > > >
> > > > > > > > Attached is my python script for embedding
> > > > > (python_script_gfs.py.txt).
> > > > > > I
> > > > > > > am
> > > > > > > > using PYGRIB library to handle the grib format.
> > > > > > > > When I run this script just as a python, it basically
works.
> It
> > > > > > extracts
> > > > > > > a
> > > > > > > > given variables into NUMPY array and give some plot.
> > > > > > > > When I use this as python embedding in MET, it gives
> > > "Segmentation
> > > > > > fault
> > > > > > > > (core dumped)" at Line 47. This line extracts the data
values
> > of
> > > > > given
> > > > > > > Grib
> > > > > > > > Record from Grib file, and returns as a NUMPY array. I
have
> > used
> > > > very
> > > > > > > basic
> > > > > > > > MET executable to test this.
> > > > > > > > $ plot_data_plane PYTHON_NUMPY test.ps
> > > > > 'name="python_script_gfs.py";'
> > > > > > > >
> > > > > > > > I am testing this issue on NCAR Cheyenne machine with
> following
> > > > > > > > environment.
> > > > > > > >
> > > > > > > > $ module purge
> > > > > > > >
> > > > > > > > $ module use
/glade/p/ral/jntp/MET/MET_releases/modulefiles
> > > > > > > >
> > > > > > > > $ module load met/8.0
> > > > > > > >
> > > > > > > > $ ncar_pylib
> > > > > > > >
> > > > > > > > FYI, I can successfully use python embedding of NetCDF
files
> > for
> > > > > > > > plot_data_plane or grid_stat.
> > > > > > > >
> > > > > > > > Thank you for your help!
> > > > > > > > BJ
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > >
> > > > > > > > Byoung-Joo (BJ) Jung
> > > > > > > >
> > > > > > > > National Center for Atmospheric Research
> > > > > > > >
> > > > > > > > Mesoscale and Microscale Meteorology Laboratory
> > > > > > > >
> > > > > > > > PO Box 3000, Boulder CO 80307
> > > > > > > >
> > > > > > > > Phone: 303-497-8151
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Byoung-Joo (BJ) Jung
> > > > > >
> > > > > > National Center for Atmospheric Research
> > > > > >
> > > > > > Mesoscale and Microscale Meteorology Laboratory
> > > > > >
> > > > > > PO Box 3000, Boulder CO 80307
> > > > > >
> > > > > > Phone: 303-497-8151
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > > --
> > > >
> > > > Byoung-Joo (BJ) Jung
> > > >
> > > > National Center for Atmospheric Research
> > > >
> > > > Mesoscale and Microscale Meteorology Laboratory
> > > >
> > > > PO Box 3000, Boulder CO 80307
> > > >
> > > > Phone: 303-497-8151
> > > >
> > > >
> > >
> > >
> >
> > --
> >
> > Byoung-Joo (BJ) Jung
> >
> > National Center for Atmospheric Research
> >
> > Mesoscale and Microscale Meteorology Laboratory
> >
> > PO Box 3000, Boulder CO 80307
> >
> > Phone: 303-497-8151
> >
> >
>
>
--
Byoung-Joo (BJ) Jung
National Center for Atmospheric Research
Mesoscale and Microscale Meteorology Laboratory
PO Box 3000, Boulder CO 80307
Phone: 303-497-8151
------------------------------------------------
Subject: Python embedding for grib format
From: Julie Prestopnik
Time: Fri Apr 26 10:02:57 2019
Hi BJ.
I made some changes on Cheyenne. Could you please try running the
following?
module load python/2.7.14
ncar_pylib
Once in the ncar_pylib environment, please run:
module use /glade/p/ral/jntp/MET/MET_releases/modulefiles
module load met/8.0_python
Please note the addition of the "_python" in loading the module file
for
met-8.0.
Then, please retry running your script and let me know if it works for
you?
Thank you for your patience while we try to resolve this issue.
Julie
On Thu, Apr 18, 2019 at 4:46 PM bjung at ucar.edu via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704 >
>
> Thanks for your efforts, Julie.
>
> BJ
>
> On Thu, Apr 18, 2019 at 3:45 PM Julie Prestopnik via RT
<met_help at ucar.edu
> >
> wrote:
>
> > >
> > > Can we try to *build* met v8 with *python/2.7.14* and try to
reproduce
> my
> > > original error ?
> >
> >
> > Yes. When I loaded pygrib, I saw Python 2.7.13 being used, so I
rebuilt
> > met-8.0 using Python 2.7.13 (it had been using 2.7.14). I will
rebuild
> > using 2.7.14 and we'll see if we can reproduce your original
error. I
> may
> > not be able do to this until next week however as I will be out of
the
> > office most of tomorrow.
> >
> > I will follow up once I have reinstalled met-8.0
> >
> > Julie
> >
> > On Thu, Apr 18, 2019 at 1:21 PM bjung at ucar.edu via RT
<met_help at ucar.edu
> >
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704 >
> > >
> > > Hi Julie,
> > >
> > > First, "ncar_pylib" is an environment to load "useful" python
> libraries.
> > > You can see the information here:
> > >
> > >
> >
> https://www2.cisl.ucar.edu/resources/computational-
systems/cheyenne/software/python
> > >
> > > I have ask CISL whether they have changed or removed
"ncar_pylib"
> > > capabilities with Python/2.7.13, and they replied as below.
> > >
> > > > Thanks for the email. The "ncar_pylib" virtual environments
have
> never
> > > been available in Python
> > > > 2.7.13. We introduced that capability with 2.7.14, and it is
> available
> > in
> > > 2.7.15 as well. It's
> > > > possible that you were activating the virtual environments in
some
> > other
> > > way, but the only
> > > > supported versions are 2.7.14, 2.7.15, 3.6.4, and 3.6.8 (the
current
> > > default). I would recommend
> > > > that you switch to one of those versions of Python to use the
NPL.
> > >
> > >
> > > I can not remember which python version was used to build the
> "pre-built"
> > > of met v8 on Cheyenne (Which was working until April 16th).
> > > But *currently*, that seems to be *python/2.7.13* . The met
modules &
> > > binary files are refreshed on April 17.
> > > Also, basic python embedding does not work with these new binary
files.
> > >
> > > Can we try to *build* met v8 with *python/2.7.14* and try to
reproduce
> my
> > > original error ?
> > >
> > > Thanks,
> > > BJ
> > >
> > > cc: Craig Schwartz
> > >
> > >
> > >
> > > On Wed, Apr 17, 2019 at 2:46 PM Julie Prestopnik via RT <
> > met_help at ucar.edu
> > > >
> > > wrote:
> > >
> > > > Thank you. I'll wait to hear back from you.
> > > >
> > > > Julie
> > > >
> > > > On Wed, Apr 17, 2019 at 2:01 PM bjung at ucar.edu via RT <
> > met_help at ucar.edu
> > > >
> > > > wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704 >
> > > > >
> > > > > Hi Julie,
> > > > > Thanks for having a look at this issue.
> > > > >
> > > > > Just tried it, but it also fails for me to run "ncar_pylib"
with
> > above
> > > > > configuration.
> > > > > I will figure out what happened , and whether there is other
way to
> > > load
> > > > > Python Libraries properly later today.
> > > > >
> > > > > Thanks,
> > > > > BJ
> > > > >
> > > > > On Wed, Apr 17, 2019 at 12:41 PM Julie Prestopnik via RT <
> > > > > met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > > Hi BJ.
> > > > > >
> > > > > > I made a change to the compilation of met-8.0. I am
attempting
> to
> > > test
> > > > > > with the same environment as you, but am having
difficulty. For
> > > > example,
> > > > > > running the commands that you described in your original
email
> > > doesn't
> > > > > > quite work for me:
> > > > > >
> > > > > > cheyenne1:jpresto> module purge
> > > > > > > cheyenne1:jpresto> module use
> > > > > > > /glade/p/ral/jntp/MET/MET_releases/modulefiles
> > > > > > > cheyenne1:jpresto> module load met/8.0
> > > > > > > cheyenne1:jpresto> ncar_pylib
> > > > > > > ncar_pylib: Command not found.
> > > > > > >
> > > > > >
> > > > > > I found that I could load "pygrib" by doing the following:
> > > > > >
> > > > > > > cheyenne1:jpresto> module load python/2.7.13
> > > > > > > cheyenne1:jpresto> module load numpy
> > > > > > > cheyenne1:jpresto> module load pyproj
> > > > > > > cheyenne1:jpresto> module load pygrib
> > > > > > > cheyenne1:jpresto> module use
> > > > > > > /glade/p/ral/jntp/MET/MET_releases/modulefiles
> > > > > > > cheyenne1:jpresto> module load met/8.0
> > > > > > > cheyenne1:jpresto> ncar_pylib
> > > > > > > ncar_pylib: Command not found.
> > > > > > >
> > > > > >
> > > > > > I am not familiar with ncar_pylib, so perhaps I am just
ignorant
> > > > > regarding
> > > > > > what to do here.
> > > > > >
> > > > > > Could you please retry running your script in a clean
shell so
> that
> > > the
> > > > > > changes I made will be picked up when you load met-8.0 and
let me
> > > know
> > > > if
> > > > > > it works for you?
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > Julie
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Mon, Apr 15, 2019 at 1:20 PM bjung at ucar.edu via RT <
> > > > met_help at ucar.edu
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > >
> > > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704
> >
> > > > > > >
> > > > > > > John,
> > > > > > >
> > > > > > > Yes. that could be related to environmental
configuration.
> > > > > > > Thanks for having a look at this issue !
> > > > > > >
> > > > > > > BJ
> > > > > > >
> > > > > > >
> > > > > > > On Mon, Apr 15, 2019 at 1:04 PM John Halley Gotway via
RT <
> > > > > > > met_help at ucar.edu>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi BJ,
> > > > > > > >
> > > > > > > > I see you're having trouble running your python script
> through
> > > MET
> > > > on
> > > > > > > > cheyenne. I tried it myself and see the same behavior
you
> > > > describe.
> > > > > > I'm
> > > > > > > > going to assign this ticket to Julie Prestopnik, the
person
> who
> > > > > > compiles
> > > > > > > > MET on many platforms, including cheyenne. I'm
wondering how
> > she
> > > > > > > > configured MET on cheyenne.
> > > > > > > >
> > > > > > > > I suspect it was not using the ncar_pylib environment.
> > Perhaps,
> > > > she
> > > > > > > could
> > > > > > > > try recompiling MET to be consistent with that
environment to
> > see
> > > > if
> > > > > > that
> > > > > > > > fixes this issue?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > John
> > > > > > > >
> > > > > > > > On Fri, Apr 12, 2019 at 11:42 AM bjung at ucar.edu via RT
<
> > > > > > > met_help at ucar.edu>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > >
> > > > > > > > > Fri Apr 12 11:37:05 2019: Request 89704 was acted
upon.
> > > > > > > > > Transaction: Ticket created by bjung at ucar.edu
> > > > > > > > > Queue: met_help
> > > > > > > > > Subject: Python embedding for grib format
> > > > > > > > > Owner: Nobody
> > > > > > > > > Requestors: bjung at ucar.edu
> > > > > > > > > Status: new
> > > > > > > > > Ticket <URL:
> > > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704
> > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Hi MET-Help,
> > > > > > > > >
> > > > > > > > > I am trying to use python embedding for grib format
in MET
> > v8.
> > > > > > > > > I know that MET can already handle Grib files, but
python
> > > > embedding
> > > > > > > would
> > > > > > > > > give more flexibility.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Attached is my python script for embedding
> > > > > > (python_script_gfs.py.txt).
> > > > > > > I
> > > > > > > > am
> > > > > > > > > using PYGRIB library to handle the grib format.
> > > > > > > > > When I run this script just as a python, it
basically
> works.
> > It
> > > > > > > extracts
> > > > > > > > a
> > > > > > > > > given variables into NUMPY array and give some plot.
> > > > > > > > > When I use this as python embedding in MET, it gives
> > > > "Segmentation
> > > > > > > fault
> > > > > > > > > (core dumped)" at Line 47. This line extracts the
data
> values
> > > of
> > > > > > given
> > > > > > > > Grib
> > > > > > > > > Record from Grib file, and returns as a NUMPY array.
I have
> > > used
> > > > > very
> > > > > > > > basic
> > > > > > > > > MET executable to test this.
> > > > > > > > > $ plot_data_plane PYTHON_NUMPY test.ps
> > > > > > 'name="python_script_gfs.py";'
> > > > > > > > >
> > > > > > > > > I am testing this issue on NCAR Cheyenne machine
with
> > following
> > > > > > > > > environment.
> > > > > > > > >
> > > > > > > > > $ module purge
> > > > > > > > >
> > > > > > > > > $ module use
/glade/p/ral/jntp/MET/MET_releases/modulefiles
> > > > > > > > >
> > > > > > > > > $ module load met/8.0
> > > > > > > > >
> > > > > > > > > $ ncar_pylib
> > > > > > > > >
> > > > > > > > > FYI, I can successfully use python embedding of
NetCDF
> files
> > > for
> > > > > > > > > plot_data_plane or grid_stat.
> > > > > > > > >
> > > > > > > > > Thank you for your help!
> > > > > > > > > BJ
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > >
> > > > > > > > > Byoung-Joo (BJ) Jung
> > > > > > > > >
> > > > > > > > > National Center for Atmospheric Research
> > > > > > > > >
> > > > > > > > > Mesoscale and Microscale Meteorology Laboratory
> > > > > > > > >
> > > > > > > > > PO Box 3000, Boulder CO 80307
> > > > > > > > >
> > > > > > > > > Phone: 303-497-8151
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > >
> > > > > > > Byoung-Joo (BJ) Jung
> > > > > > >
> > > > > > > National Center for Atmospheric Research
> > > > > > >
> > > > > > > Mesoscale and Microscale Meteorology Laboratory
> > > > > > >
> > > > > > > PO Box 3000, Boulder CO 80307
> > > > > > >
> > > > > > > Phone: 303-497-8151
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Byoung-Joo (BJ) Jung
> > > > >
> > > > > National Center for Atmospheric Research
> > > > >
> > > > > Mesoscale and Microscale Meteorology Laboratory
> > > > >
> > > > > PO Box 3000, Boulder CO 80307
> > > > >
> > > > > Phone: 303-497-8151
> > > > >
> > > > >
> > > >
> > > >
> > >
> > > --
> > >
> > > Byoung-Joo (BJ) Jung
> > >
> > > National Center for Atmospheric Research
> > >
> > > Mesoscale and Microscale Meteorology Laboratory
> > >
> > > PO Box 3000, Boulder CO 80307
> > >
> > > Phone: 303-497-8151
> > >
> > >
> >
> >
>
> --
>
> Byoung-Joo (BJ) Jung
>
> National Center for Atmospheric Research
>
> Mesoscale and Microscale Meteorology Laboratory
>
> PO Box 3000, Boulder CO 80307
>
> Phone: 303-497-8151
>
>
------------------------------------------------
Subject: Python embedding for grib format
From: bjung at ucar.edu
Time: Fri Apr 26 16:24:16 2019
Hi Julie,
Thanks for your work!
First of all, recently I can use "python_embedding" for NetCDF again,
with
pre-installed *met/8.0* module. My original issue was that I could not
use
"python_embedding" for *Grib2* format (I tried this with pygrib
library,
which is installed as NCAR Python Package: *ncar_pylib* ).
Then, I followed your instruction, but python_embedding does not work
(even
for NetCDF format). I have added below what I tried. I also checked
"python-config" for each stage. These "python-config" seems to change
at
each stage...
First, this is the default module list when I login on Cheyenne.
% module li
Currently Loaded Modules:
1) ncarenv/1.2 2) intel/17.0.1 3) ncarcompilers/0.4.1 4)
mpt/2.19
5) netcdf/4.6.1 6) python/2.7.14
% python-config --cflags
-I/gpfs/u/apps/ch/opt/python/2.7.14/intel/17.0.1/pkg-
library/20180329/include/python2.7
-I/gpfs/u/apps/ch/opt/python/2.7.14/intel/17.0.1/pkg-
library/20180329/include/python2.7
-fno-strict-aliasing -fp-model strict -g -O2 -DNDEBUG -g -O3 -Wall
-Wstrict-prototypes
% python-config --ldflags
-L/glade/u/apps/ch/opt/python/2.7.14/intel/17.0.1/lib/python2.7/config
-lpython2.7 -lpthread -ldl -lutil -lm -Xlinker -export-dynamic
% module use /glade/p/ral/jntp/MET/MET_releases/modulefiles
% python-config --cflags
-I/gpfs/u/apps/ch/opt/python/2.7.14/intel/17.0.1/pkg-
library/20180329/include/python2.7
-I/gpfs/u/apps/ch/opt/python/2.7.14/intel/17.0.1/pkg-
library/20180329/include/python2.7
-fno-strict-aliasing -fp-model strict -g -O2 -DNDEBUG -g -O3 -Wall
-Wstrict-prototypes
% python-config --ldflags
-L/glade/u/apps/ch/opt/python/2.7.14/intel/17.0.1/lib/python2.7/config
-lpython2.7 -lpthread -ldl -lutil -lm -Xlinker -export-dynamic
% module load met/8.0_python
% python-config --cflags
-I/gpfs/u/apps/ch/opt/python/2.7.14/intel/17.0.1/include/python2.7
-I/gpfs/u/apps/ch/opt/python/2.7.14/intel/17.0.1/include/python2.7
-fno-strict-aliasing -fp-model strict -g -O2 -DNDEBUG -g -O3 -Wall
-Wstrict-prototypes
% python-config --ldflags
-L/glade/u/apps/ch/opt/python/2.7.14/intel/17.0.1/lib/python2.7/config
-lpython2.7 -lpthread -ldl -lutil -lm -Xlinker -export-dynamic
Then I run my script and got error below...
CALLING: /glade/p/ral/jntp/MET/MET_releases/8.0/bin//grid_stat
/glade/scratch/bjung/met/draft_2nd/FCST/2018110100/gfs.0p25.2018110100.f006.grib2
PYTHON_NUMPY
/glade/scratch/bjung/met/draft_2nd/met_config/GridStatConfig_all
-outdir .
-v 2
DEBUG 1: Default Config File:
/glade/p/ral/jntp/MET/MET_releases/8.0/share/met/config/GridStatConfig_default
DEBUG 1: User Config File:
/glade/scratch/bjung/met/draft_2nd/met_config/GridStatConfig_all
ERROR :
ERROR : python_dataplane() -> an error occurred importing module
"./python_script_obs.py"
ERROR :
Traceback (most recent call last):
File
"/gpfs/fs1/scratch/bjung/met/draft_2nd/working/python_script_obs.py",
line
2, in <module>
import numpy as np
ImportError: No module named numpy
DEBUG 1: Opening data file: PYTHON_NUMPY
ERROR :
ERROR : python_dataplane() -> an error occurred importing module
"./python_script_obs.py"
ERROR :
Traceback (most recent call last):
File
"/gpfs/fs1/scratch/bjung/met/draft_2nd/working/python_script_obs.py",
line
2, in <module>
import numpy as np
ImportError: No module named numpy
ERROR: For GFS, /glade/p/ral/jntp/MET/MET_releases/8.0/bin//grid_stat
crashed Exit status: 1
Thank you for your investigation!!
BJ
On Fri, Apr 26, 2019 at 10:03 AM Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:
> Hi BJ.
>
> I made some changes on Cheyenne. Could you please try running the
> following?
>
> module load python/2.7.14
> ncar_pylib
>
> Once in the ncar_pylib environment, please run:
>
> module use /glade/p/ral/jntp/MET/MET_releases/modulefiles
> module load met/8.0_python
>
> Please note the addition of the "_python" in loading the module file
for
> met-8.0.
>
> Then, please retry running your script and let me know if it works
for you?
>
> Thank you for your patience while we try to resolve this issue.
>
> Julie
>
>
> On Thu, Apr 18, 2019 at 4:46 PM bjung at ucar.edu via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704 >
> >
> > Thanks for your efforts, Julie.
> >
> > BJ
> >
> > On Thu, Apr 18, 2019 at 3:45 PM Julie Prestopnik via RT <
> met_help at ucar.edu
> > >
> > wrote:
> >
> > > >
> > > > Can we try to *build* met v8 with *python/2.7.14* and try to
> reproduce
> > my
> > > > original error ?
> > >
> > >
> > > Yes. When I loaded pygrib, I saw Python 2.7.13 being used, so I
> rebuilt
> > > met-8.0 using Python 2.7.13 (it had been using 2.7.14). I will
rebuild
> > > using 2.7.14 and we'll see if we can reproduce your original
error. I
> > may
> > > not be able do to this until next week however as I will be out
of the
> > > office most of tomorrow.
> > >
> > > I will follow up once I have reinstalled met-8.0
> > >
> > > Julie
> > >
> > > On Thu, Apr 18, 2019 at 1:21 PM bjung at ucar.edu via RT <
> met_help at ucar.edu
> > >
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704
>
> > > >
> > > > Hi Julie,
> > > >
> > > > First, "ncar_pylib" is an environment to load "useful" python
> > libraries.
> > > > You can see the information here:
> > > >
> > > >
> > >
> >
> https://www2.cisl.ucar.edu/resources/computational-
systems/cheyenne/software/python
> > > >
> > > > I have ask CISL whether they have changed or removed
"ncar_pylib"
> > > > capabilities with Python/2.7.13, and they replied as below.
> > > >
> > > > > Thanks for the email. The "ncar_pylib" virtual environments
have
> > never
> > > > been available in Python
> > > > > 2.7.13. We introduced that capability with 2.7.14, and it is
> > available
> > > in
> > > > 2.7.15 as well. It's
> > > > > possible that you were activating the virtual environments
in some
> > > other
> > > > way, but the only
> > > > > supported versions are 2.7.14, 2.7.15, 3.6.4, and 3.6.8 (the
> current
> > > > default). I would recommend
> > > > > that you switch to one of those versions of Python to use
the NPL.
> > > >
> > > >
> > > > I can not remember which python version was used to build the
> > "pre-built"
> > > > of met v8 on Cheyenne (Which was working until April 16th).
> > > > But *currently*, that seems to be *python/2.7.13* . The met
modules &
> > > > binary files are refreshed on April 17.
> > > > Also, basic python embedding does not work with these new
binary
> files.
> > > >
> > > > Can we try to *build* met v8 with *python/2.7.14* and try to
> reproduce
> > my
> > > > original error ?
> > > >
> > > > Thanks,
> > > > BJ
> > > >
> > > > cc: Craig Schwartz
> > > >
> > > >
> > > >
> > > > On Wed, Apr 17, 2019 at 2:46 PM Julie Prestopnik via RT <
> > > met_help at ucar.edu
> > > > >
> > > > wrote:
> > > >
> > > > > Thank you. I'll wait to hear back from you.
> > > > >
> > > > > Julie
> > > > >
> > > > > On Wed, Apr 17, 2019 at 2:01 PM bjung at ucar.edu via RT <
> > > met_help at ucar.edu
> > > > >
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704 >
> > > > > >
> > > > > > Hi Julie,
> > > > > > Thanks for having a look at this issue.
> > > > > >
> > > > > > Just tried it, but it also fails for me to run
"ncar_pylib" with
> > > above
> > > > > > configuration.
> > > > > > I will figure out what happened , and whether there is
other way
> to
> > > > load
> > > > > > Python Libraries properly later today.
> > > > > >
> > > > > > Thanks,
> > > > > > BJ
> > > > > >
> > > > > > On Wed, Apr 17, 2019 at 12:41 PM Julie Prestopnik via RT <
> > > > > > met_help at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi BJ.
> > > > > > >
> > > > > > > I made a change to the compilation of met-8.0. I am
attempting
> > to
> > > > test
> > > > > > > with the same environment as you, but am having
difficulty.
> For
> > > > > example,
> > > > > > > running the commands that you described in your original
email
> > > > doesn't
> > > > > > > quite work for me:
> > > > > > >
> > > > > > > cheyenne1:jpresto> module purge
> > > > > > > > cheyenne1:jpresto> module use
> > > > > > > > /glade/p/ral/jntp/MET/MET_releases/modulefiles
> > > > > > > > cheyenne1:jpresto> module load met/8.0
> > > > > > > > cheyenne1:jpresto> ncar_pylib
> > > > > > > > ncar_pylib: Command not found.
> > > > > > > >
> > > > > > >
> > > > > > > I found that I could load "pygrib" by doing the
following:
> > > > > > >
> > > > > > > > cheyenne1:jpresto> module load python/2.7.13
> > > > > > > > cheyenne1:jpresto> module load numpy
> > > > > > > > cheyenne1:jpresto> module load pyproj
> > > > > > > > cheyenne1:jpresto> module load pygrib
> > > > > > > > cheyenne1:jpresto> module use
> > > > > > > > /glade/p/ral/jntp/MET/MET_releases/modulefiles
> > > > > > > > cheyenne1:jpresto> module load met/8.0
> > > > > > > > cheyenne1:jpresto> ncar_pylib
> > > > > > > > ncar_pylib: Command not found.
> > > > > > > >
> > > > > > >
> > > > > > > I am not familiar with ncar_pylib, so perhaps I am just
> ignorant
> > > > > > regarding
> > > > > > > what to do here.
> > > > > > >
> > > > > > > Could you please retry running your script in a clean
shell so
> > that
> > > > the
> > > > > > > changes I made will be picked up when you load met-8.0
and let
> me
> > > > know
> > > > > if
> > > > > > > it works for you?
> > > > > > >
> > > > > > > Thanks!
> > > > > > >
> > > > > > > Julie
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Mon, Apr 15, 2019 at 1:20 PM bjung at ucar.edu via RT <
> > > > > met_help at ucar.edu
> > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704
> > >
> > > > > > > >
> > > > > > > > John,
> > > > > > > >
> > > > > > > > Yes. that could be related to environmental
configuration.
> > > > > > > > Thanks for having a look at this issue !
> > > > > > > >
> > > > > > > > BJ
> > > > > > > >
> > > > > > > >
> > > > > > > > On Mon, Apr 15, 2019 at 1:04 PM John Halley Gotway via
RT <
> > > > > > > > met_help at ucar.edu>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi BJ,
> > > > > > > > >
> > > > > > > > > I see you're having trouble running your python
script
> > through
> > > > MET
> > > > > on
> > > > > > > > > cheyenne. I tried it myself and see the same
behavior you
> > > > > describe.
> > > > > > > I'm
> > > > > > > > > going to assign this ticket to Julie Prestopnik, the
person
> > who
> > > > > > > compiles
> > > > > > > > > MET on many platforms, including cheyenne. I'm
wondering
> how
> > > she
> > > > > > > > > configured MET on cheyenne.
> > > > > > > > >
> > > > > > > > > I suspect it was not using the ncar_pylib
environment.
> > > Perhaps,
> > > > > she
> > > > > > > > could
> > > > > > > > > try recompiling MET to be consistent with that
environment
> to
> > > see
> > > > > if
> > > > > > > that
> > > > > > > > > fixes this issue?
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > John
> > > > > > > > >
> > > > > > > > > On Fri, Apr 12, 2019 at 11:42 AM bjung at ucar.edu via
RT <
> > > > > > > > met_help at ucar.edu>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Fri Apr 12 11:37:05 2019: Request 89704 was acted
upon.
> > > > > > > > > > Transaction: Ticket created by bjung at ucar.edu
> > > > > > > > > > Queue: met_help
> > > > > > > > > > Subject: Python embedding for grib format
> > > > > > > > > > Owner: Nobody
> > > > > > > > > > Requestors: bjung at ucar.edu
> > > > > > > > > > Status: new
> > > > > > > > > > Ticket <URL:
> > > > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89704
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Hi MET-Help,
> > > > > > > > > >
> > > > > > > > > > I am trying to use python embedding for grib
format in
> MET
> > > v8.
> > > > > > > > > > I know that MET can already handle Grib files, but
python
> > > > > embedding
> > > > > > > > would
> > > > > > > > > > give more flexibility.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Attached is my python script for embedding
> > > > > > > (python_script_gfs.py.txt).
> > > > > > > > I
> > > > > > > > > am
> > > > > > > > > > using PYGRIB library to handle the grib format.
> > > > > > > > > > When I run this script just as a python, it
basically
> > works.
> > > It
> > > > > > > > extracts
> > > > > > > > > a
> > > > > > > > > > given variables into NUMPY array and give some
plot.
> > > > > > > > > > When I use this as python embedding in MET, it
gives
> > > > > "Segmentation
> > > > > > > > fault
> > > > > > > > > > (core dumped)" at Line 47. This line extracts the
data
> > values
> > > > of
> > > > > > > given
> > > > > > > > > Grib
> > > > > > > > > > Record from Grib file, and returns as a NUMPY
array. I
> have
> > > > used
> > > > > > very
> > > > > > > > > basic
> > > > > > > > > > MET executable to test this.
> > > > > > > > > > $ plot_data_plane PYTHON_NUMPY test.ps
> > > > > > > 'name="python_script_gfs.py";'
> > > > > > > > > >
> > > > > > > > > > I am testing this issue on NCAR Cheyenne machine
with
> > > following
> > > > > > > > > > environment.
> > > > > > > > > >
> > > > > > > > > > $ module purge
> > > > > > > > > >
> > > > > > > > > > $ module use
> /glade/p/ral/jntp/MET/MET_releases/modulefiles
> > > > > > > > > >
> > > > > > > > > > $ module load met/8.0
> > > > > > > > > >
> > > > > > > > > > $ ncar_pylib
> > > > > > > > > >
> > > > > > > > > > FYI, I can successfully use python embedding of
NetCDF
> > files
> > > > for
> > > > > > > > > > plot_data_plane or grid_stat.
> > > > > > > > > >
> > > > > > > > > > Thank you for your help!
> > > > > > > > > > BJ
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > >
> > > > > > > > > > Byoung-Joo (BJ) Jung
> > > > > > > > > >
> > > > > > > > > > National Center for Atmospheric Research
> > > > > > > > > >
> > > > > > > > > > Mesoscale and Microscale Meteorology Laboratory
> > > > > > > > > >
> > > > > > > > > > PO Box 3000, Boulder CO 80307
> > > > > > > > > >
> > > > > > > > > > Phone: 303-497-8151
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > >
> > > > > > > > Byoung-Joo (BJ) Jung
> > > > > > > >
> > > > > > > > National Center for Atmospheric Research
> > > > > > > >
> > > > > > > > Mesoscale and Microscale Meteorology Laboratory
> > > > > > > >
> > > > > > > > PO Box 3000, Boulder CO 80307
> > > > > > > >
> > > > > > > > Phone: 303-497-8151
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Byoung-Joo (BJ) Jung
> > > > > >
> > > > > > National Center for Atmospheric Research
> > > > > >
> > > > > > Mesoscale and Microscale Meteorology Laboratory
> > > > > >
> > > > > > PO Box 3000, Boulder CO 80307
> > > > > >
> > > > > > Phone: 303-497-8151
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > > --
> > > >
> > > > Byoung-Joo (BJ) Jung
> > > >
> > > > National Center for Atmospheric Research
> > > >
> > > > Mesoscale and Microscale Meteorology Laboratory
> > > >
> > > > PO Box 3000, Boulder CO 80307
> > > >
> > > > Phone: 303-497-8151
> > > >
> > > >
> > >
> > >
> >
> > --
> >
> > Byoung-Joo (BJ) Jung
> >
> > National Center for Atmospheric Research
> >
> > Mesoscale and Microscale Meteorology Laboratory
> >
> > PO Box 3000, Boulder CO 80307
> >
> > Phone: 303-497-8151
> >
> >
>
>
--
Byoung-Joo (BJ) Jung
National Center for Atmospheric Research
Mesoscale and Microscale Meteorology Laboratory
PO Box 3000, Boulder CO 80307
Phone: 303-497-8151
------------------------------------------------
More information about the Met_help
mailing list