如何构建和分发依赖第三方libFoo.so的PythonCython软件包

如何构建和分发依赖第三方libFoo.so的Python / Cython软件包

您可能知道，建议使用分发格式来分发带有已编译组件的Python模块。似乎没有任何标准的跨平台方式将第三方本机库捆绑到一个轮子中。但是，有用于此目的的特定于平台的工具。

在Linux上，使用

auditwheel

。

auditwheel

修改现有的Linux wheel文件以添加基本的“
manylinux ”标准中未包含的任何第三方库。以下是在全新安装的Ubuntu
17.10上如何将其与项目一起使用的演练：

首先，安装基本的Python开发工具，以及带有其标头的第三方库：

root@ubuntu-17:~# apt-get install cython python-pip unziproot@ubuntu-17:~# apt-get install libsundials-serial-dev

然后将您的项目构建到wheel文件中：

root@ubuntu-17:~# cd cython-example/root@ubuntu-17:~/cython-example# python setup.py bdist_wheel[...]root@ubuntu-17:~/cython-example# cd dist/root@ubuntu-17:~/cython-example/dist# lltotal 80drwxr-xr-x 2 root root  4096 Nov  8 11:28 ./drwxr-xr-x 7 root root  4096 Nov  8 11:28 ../-rw-r--r-- 1 root root 70135 Nov  8 11:28 poc-0.0.0-cp27-cp27mu-linux_x86_64.whlroot@ubuntu-17:~/cython-example/dist# unzip -l poc-0.0.0-cp27-cp27mu-linux_x86_64.whlArchive:  poc-0.0.0-cp27-cp27mu-linux_x86_64.whl  Length      Date    Time    Name---------  ---------- -----   ----    62440  2017-11-08 11:28   poc/do_stuff.so        2  2017-11-08 11:28   poc/__init__.py   116648  2017-11-08 11:28   poc/cython_extensions/helloworld.so        2  2017-11-08 11:28   poc/cython_extensions/__init__.py       10  2017-11-08 11:28   poc-0.0.0.dist-info/DEscriptION.rst      211  2017-11-08 11:28   poc-0.0.0.dist-info/metadata.json        4  2017-11-08 11:28   poc-0.0.0.dist-info/top_level.txt      105  2017-11-08 11:28   poc-0.0.0.dist-info/WHEEL      167  2017-11-08 11:28   poc-0.0.0.dist-info/metaDATA      793  2017-11-08 11:28   poc-0.0.0.dist-info/RECORD---------          -------   180382          10 files

wheel文件现在可以在本地安装并经过测试：

root@ubuntu-17:~/cython-example/dist# pip install poc-0.0.0-cp27-cp27mu-linux_x86_64.whl[...]root@ubuntu-17:~/cython-example/dist# python -c "from poc.do_stuff import hello; hello()"hello cython0.841470984808trying to load the sundials program3-species kinetics problemAt t = 2.6391e-01      y =  9.899653e-01    3.470564e-05    1.000000e-02    rootsfound[] =   0   1At t = 4.0000e-01      y =  9.851641e-01    3.386242e-05    1.480205e-02[...]

现在我们安装该

auditwheel

工具。它需要Python 3，但是能够处理Python 2或3的轮子。

root@ubuntu-17:~/cython-example/dist# apt-get install python3-piproot@ubuntu-17:~/cython-example/dist# pip3 install auditwheel

auditwheel

使用另一个称为“

patchelf

工作”的工具。不幸的是，

patchelf

Ubuntu
17.10随附的版本缺少一个错误修正，没有该修正，auditwheel将无法运行。因此，我们必须从源代码构建脚本（脚本取自manylinux
Docker映像）：

root@ubuntu-17:~# apt-get install autoconfroot@ubuntu-17:~# PATCHELF_VERSION=6bfcafbba8d89e44f9ac9582493b4f27d9d8c369root@ubuntu-17:~# curl -sL -o patchelf.tar.gz https://github.com/NixOS/patchelf/archive/$PATCHELF_VERSION.tar.gzroot@ubuntu-17:~# tar -xzf patchelf.tar.gzroot@ubuntu-17:~# (cd patchelf-$PATCHELF_VERSION && ./bootstrap.sh && ./configure && make && make install)

现在，我们可以检查方向盘需要哪些第三方库：

root@ubuntu-17:~/cython-example/dist# auditwheel show poc-0.0.0-cp27-cp27mu-linux_x86_64.whlpoc-0.0.0-cp27-cp27mu-linux_x86_64.whl is consistent with thefollowing platform tag: "linux_x86_64".The wheel references external versioned symbols in these system-provided shared libraries: libc.so.6 with versions {'GLIBC_2.4','GLIBC_2.2.5', 'GLIBC_2.3.4'}The following external shared libraries are required by the wheel:{    "libblas.so.3": "/usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1",    "libc.so.6": "/lib/x86_64-linux-gnu/libc-2.26.so",    "libgcc_s.so.1": "/lib/x86_64-linux-gnu/libgcc_s.so.1",    "libgfortran.so.4": "/usr/lib/x86_64-linux-gnu/libgfortran.so.4.0.0",    "liblapack.so.3": "/usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1",    "libm.so.6": "/lib/x86_64-linux-gnu/libm-2.26.so",    "libpthread.so.0": "/lib/x86_64-linux-gnu/libpthread-2.26.so",    "libquadmath.so.0": "/usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0",    "libsundials_cvodes.so.2": "/usr/lib/libsundials_cvodes.so.2.0.0",    "libsundials_nvecserial.so.0": "/usr/lib/libsundials_nvecserial.so.0.0.2"}In order to achieve the tag platform tag "manylinux1_x86_64" thefollowing shared library dependencies will need to be eliminated:libblas.so.3, libgfortran.so.4, liblapack.so.3, libquadmath.so.0,libsundials_cvodes.so.2, libsundials_nvecserial.so.0

并创建一个捆绑它们的新轮子：

root@ubuntu-17:~/cython-example/dist# auditwheel repair poc-0.0.0-cp27-cp27mu-linux_x86_64.whlRepairing poc-0.0.0-cp27-cp27mu-linux_x86_64.whlGrafting: /usr/lib/libsundials_nvecserial.so.0.0.2 -> poc/.libs/libsundials_nvecserial-42b4120e.so.0.0.2Grafting: /usr/lib/libsundials_cvodes.so.2.0.0 -> poc/.libs/libsundials_cvodes-50fde5ee.so.2.0.0Grafting: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1 -> poc/.libs/liblapack-549933c4.so.3.7.1Grafting: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1 -> poc/.libs/libblas-52fa99c8.so.3.7.1Grafting: /usr/lib/x86_64-linux-gnu/libgfortran.so.4.0.0 -> poc/.libs/libgfortran-2df4b07d.so.4.0.0Grafting: /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 -> poc/.libs/libquadmath-0d7c3070.so.0.0.0Setting RPATH: poc/cython_extensions/helloworld.so to "$ORIGIN/../.libs"Previous filename tags: linux_x86_64New filename tags: manylinux1_x86_64Previous WHEEL info tags: cp27-cp27mu-linux_x86_64New WHEEL info tags: cp27-cp27mu-manylinux1_x86_64Fixed-up wheel written to /root/cython-example/dist/wheelhouse/poc-0.0.0-cp27-cp27mu-manylinux1_x86_64.whlroot@ubuntu-17:~/cython-example/dist# unzip -l wheelhouse/poc-0.0.0-cp27-cp27mu-manylinux1_x86_64.whlArchive:  wheelhouse/poc-0.0.0-cp27-cp27mu-manylinux1_x86_64.whl  Length      Date    Time    Name---------  ---------- -----   ----      167  2017-11-08 11:28   poc-0.0.0.dist-info/metaDATA        4  2017-11-08 11:28   poc-0.0.0.dist-info/top_level.txt       10  2017-11-08 11:28   poc-0.0.0.dist-info/DEscriptION.rst      211  2017-11-08 11:28   poc-0.0.0.dist-info/metadata.json     1400  2017-11-08 12:08   poc-0.0.0.dist-info/RECORD      110  2017-11-08 12:08   poc-0.0.0.dist-info/WHEEL    62440  2017-11-08 11:28   poc/do_stuff.so        2  2017-11-08 11:28   poc/__init__.py   131712  2017-11-08 12:08   poc/cython_extensions/helloworld.so        2  2017-11-08 11:28   poc/cython_extensions/__init__.py   230744  2017-11-08 12:08   poc/.libs/libsundials_cvodes-50fde5ee.so.2.0.0  7005072  2017-11-08 12:08   poc/.libs/liblapack-549933c4.so.3.7.1   264024  2017-11-08 12:08   poc/.libs/libquadmath-0d7c3070.so.0.0.0  2039960  2017-11-08 12:08   poc/.libs/libgfortran-2df4b07d.so.4.0.0    17736  2017-11-08 12:08   poc/.libs/libsundials_nvecserial-42b4120e.so.0.0.2   452432  2017-11-08 12:08   poc/.libs/libblas-52fa99c8.so.3.7.1---------          ------- 10206026          16 files

如果我们卸载第三方库，则先前安装的滚轮将停止工作：

root@ubuntu-17:~/cython-example/dist# apt-get remove libsundials-serial-dev && apt-get autoremove[...]root@ubuntu-17:~/cython-example/dist# python -c "from poc.do_stuff import hello; hello()"Traceback (most recent call last):  File "<string>", line 1, in <module>  File "poc/do_stuff.pyx", line 1, in init poc.do_stuffimportError: libsundials_cvodes.so.2: cannot open shared object file: No such file or directory

但是带有捆绑库的轮子可以正常工作：

root@ubuntu-17:~/cython-example/dist# pip uninstall poc[...]root@ubuntu-17:~/cython-example/dist# pip install wheelhouse/poc-0.0.0-cp27-cp27mu-manylinux1_x86_64.whl[...]root@ubuntu-17:~/cython-example/dist# python -c "from poc.do_stuff import hello; hello()"hello cython0.841470984808trying to load the sundials program3-species kinetics problemAt t = 2.6391e-01      y =  9.899653e-01    3.470564e-05    1.000000e-02    rootsfound[] =   0   1At t = 4.0000e-01      y =  9.851641e-01    3.386242e-05    1.480205e-02[...]

在OSX上，使用

delocate

。

delocate

OSX的工作方式显然与

auditwheel

。不幸的是，我没有可用于提供演练的OSX机器。

组合示例：

使用这两种工具的一个项目是SciPy。尽管它的名称，此存储库包含所有平台（不仅仅是Mac）的官方SciPy构建过程。具体来说，将Linux构建脚本（使用

auditwheel

）和OSX构建脚本（使用

delocate

）进行比较。

要查看此过程的结果，您可能需要从PyPI下载并解压缩一些SciPy轮子。例如，

scipy-1.0.0-cp27-cp27m-manylinux1_x86_64.whl

包含以下内容：

 38513408  2017-10-25 06:02   scipy/.libs/libopenblasp-r0-39a31c03.2.18.so  1023960  2017-10-25 06:02   scipy/.libs/libgfortran-ed201abd.so.3.0.0

虽然

scipy-1.0.0-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl

包含以下内容：

   273072  2017-10-25 07:03   scipy/.dylibs/libgcc_s.1.dylib  1550456  2017-10-25 07:03   scipy/.dylibs/libgfortran.3.dylib   279932  2017-10-25 07:03   scipy/.dylibs/libquadmath.0.dylib