[pyngl-talk] Building PyNIO from source - missing file / undefined symbol

[Mike Chen]

Hi,
I tried to build the PyNio from source on CentOS 6.
A copy of Python 3.6.8 has built and installed to /opt/python-3.6.8, with
/opt/python-3.6.8/bin added to the $PATH.

Some basic dependencies are installed from yum:
> libjpeg-devel libpng-devel jasper-devel zlib-devel bzip2-devel
udunits2-devel bison

Other needed dependencies are built and installed to /opt/ncl-libs; the
/opt/ncl-libs/bin is added to the $PATH, and /opt/ncl-libs/lib added to
$LD_LIBRARY_PATH before building:

SZIP-2.1.1:
> ./configure --prefix=/opt/ncl-libs --enable-static=no

HDF5-1.8.21:
> ./configure --prefix=/opt/ncl-libs --with-szlib=/opt/ncl-libs
--enable-static=no

HDF-4.2.12:
> ./configure --prefix=/opt/ncl-libs --with-szlib=/opt/ncl-libs
--includedir=/opt/ncl-libs/include/hdf --enable-static=no
--enable-shared=yes --disable-fortran

NetCDF 4.6.1:
> ./configure --prefix=/opt/ncl-libs CPPFLAGS=-I/opt/ncl-libs/include
LDFLAGS=-L/opt/ncl-libs/lib --enable-static=no

libiconv-1.15:
> ./configure --prefix=/opt/ncl-libs --enable-static=no

libg2c-1.6.0:
> makefile: add -fPIC in CFLAGS, replace LIB=libgrib2c.a; make install

Proj4-4.9.3:
> ./configure --prefix=/opt/ncl-libs

GDAL-2.2.4:
> ./configure --prefix= /opt/ncl-libs   --with-hdf5=
/opt/ncl-libs --enable-static=no CPPFLAGS="-I /opt/ncl-libs/include -I
/opt/ncl-libs/include/hdf" LDFLAGS=-L$/opt/ncl-libs/lib --without-grib
(The GDAL seems to have a internal copy of libg2c in its source, so I
disabled this one. Doesn't seems to have negative effect on building PyNIO.)

HEFEOS-2.19v1.00:
> ./configure --prefix=/opt/ncl-libs CC=/opt/ncl-libs/bin/h4cc
--with-szlib=/opt/ncl-libs CFLAGS=-fPIC

HDFEOS5-1.15:
> ./configure --prefix=/opt/ncl-libs CC=/opt/ncl-libs/bin/h5cc
--with-szlib=/opt/ncl-libs CFLAGS=-fPIC

With the dependencies built and installed, I tried to build the PyNIO.
The source was git cloned a few days ago.
The build_py3.sh was edited:
> export CONDA_PREFIX=/opt/ncl-libs
> export HAS_SZIP=1
> add -I$CONDA_PREFIX/include/hdf in CPPFLAGS and CFLAGS
> change python setup.py build to python3 setup.py build
> change pip install . to pip3 install .
Then:
> sh build_py3.sh

The build goes well, except for failing at the final linking:
> g++ -pthread -shared -L/opt/ncl-libs/lib -D_BSD_SOURCE -D_XOPEN_SOURCE
-I/opt/ncl-libs/include -I/opt/ncl-libs/include/hdf -I/opt/ncl-libs/include
-I/opt/ncl-libs/include/hdf build/temp.linux-x86_64-3.6/niomodule.o
-Llibsrc -L/opt/ncl-libs/lib -Lbuild/temp.linux-x86_64-3.6 -lnio -ljpeg
-lpng -lz -lnetcdf -lmfhdf -ldf -lhdf5_hl -lhdf5 -lcurl -lhdfeos -lGctp
-lgrib2c -ljasper -ljpeg -lpng -lhdf5_hl -lhdf5 -lhe5_hdfeos -lGctp -lgdal
-lproj -liconv -ljasper -ljpeg -lpng -lz -lsz -lgfortran -o
build/lib.linux-x86_64-3.6/PyNIO/_nio.cpython-36m-x86_64-linux-gnu.so
>   g++: : No such file or directory

I'm not sure what file(s) was missing.
If I export the variables in build_py3.sh first and then run "python3
setup.py build", the same error appears.
If I unset those variables, copy the g++ command that failed, paste it in
terninal and run directly, the final _nio.cpython-36m-x86_64-linux-gnu.so
can be created.

Re-run the build_py3.sh will have the package installed in Python.
How ever, when I tried to import the module, the following error appears:
>>> import Nio
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/bld/pynio/Nio.py", line 83, in <module>
    from _nio import *
ImportError: /opt/python3/lib/python3.6/site-packages/PyNIO/_
nio.cpython-36m-x86_64-linux-gnu.so: undefined symbol: sd_ncinquire

I've checked the linking of the installed
/opt/python3/lib/python3.6/site-packages/PyNIO/_
nio.cpython-36m-x86_64-linux-gnu.so with ldd, and it seems that all
libraries are accounted for.

I think I'm close, but I have no idea how to deal with the undefined symbol
problem.
Any ideas?

Mike
mike.scchen at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/pyngl-talk/attachments/20190221/009328d9/attachment.html>