Commit 8f624957 authored by Kirill Smelkov's avatar Kirill Smelkov

setup: Propagate only setuptools, not whole sys.path

This updates and fixes 487e5226 (setup: specify setuptools location
explicitly when calling make.) to use @kazuhiko original idea to
propagate only setuptools location. The reason is - when propagating
whole sys.path things break under tox tests:

---- 8< ----
========================================================================== test session starts ===========================================================================
platform linux2 -- Python 2.7.12, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /home/kirr/src/wendelin/release/wendelin.core, inifile:
collected 33 items

bigarray/tests/test_arrayzodb.py .......
bigarray/tests/test_basic.py ........
bigfile/tests/test_basic.py ....
bigfile/tests/test_filefile.py .
bigfile/tests/test_filezodb.py ........
bigfile/tests/test_thread.py ....
lib/tests/test_calc.py .

======================================================================= 33 passed in 14.14 seconds =======================================================================
x86_64-linux-gnu-gcc -pthread -g -Wall -D_GNU_SOURCE -std=gnu99 -fplan9-extensions -Wno-declaration-after-statement -Wno-error=declaration-after-statement  -Iinclude -I3rdparty/ccan -I3rdparty/include   bigfile/tests/tfault.c lib/bug.c lib/utils.c 3rdparty/ccan/ccan/tap/tap.c  -o bigfile/tests/tfault.t
t/tfault-run bigfile/tests/tfault.t faultr on_pagefault
ok 1 - !pagefault_init()
Fatal Python error: Py_Initialize: Unable to get the locale encoding
  File ".../wendelin.core/.tox/py27-ZODB3-zblk0-fs-numpy110/lib/python2.7/encodings/__init__.py", line 123
    raise CodecRegistryError,\
                            ^
SyntaxError: invalid syntax

Current thread 0x00007f9b80024780 (most recent call first):
t/tfault-run: line 28: 21521 Аварийный останов         (core dumped) gdb -q -batch $tfault core > core.info
E: can't gdb(core)
Makefile:189: ошибка выполнения рецепта для цели «faultr.tfault»
make: *** [faultr.tfault] Ошибка 1
rm bigfile/tests/test_virtmem.t bigfile/tests/test_ram.t bigfile/tests/tfault.t bigfile/tests/test_pagemap.t
error: Failed to execute `make test`
ERROR: InvocationError: '.../wendelin.core/.tox/py27-ZODB3-zblk0-fs-numpy110/bin/python setup.py test'
________________________________________________________________________________ summary _________________________________________________________________________________
ERROR:   py27-ZODB3-zblk0-fs-numpy110: commands failed
---- 8< ----

What happens here is:

- gdb is used in automated tests
- gdb is linked with libpython3.5
- tox is currently running tests with python27
- setup.py sets PYTHONPATH to whole path from python27
- gdb, upon starting, initializes python runtime, which tries to load
  py27 modules under py35 -> oops.

So propagating only setuptools location practically solves the issue for
now, but still there is potential risk that in future, there will be
other modules put in the same location as setuptools (location is parent
dir of a module/package) which python tries to load on startup and which
might be incompatible between 2 & 3.

Thus not setting PYTHONPATH at all - e.g. creating environments
virtualenv way - would be better, but buildout fundamentally works the
other way. We can not also use -c 'buildout sys.path hack' in $PYTHON
itself, as $PYTHON is used in general way inside Makefile.

So let the hack with setuptools location in PYTHONPATH stay there until
it practically works.
parent 487e5226
...@@ -18,6 +18,7 @@ ...@@ -18,6 +18,7 @@
from setuptools import setup, Extension, Command, find_packages from setuptools import setup, Extension, Command, find_packages
from setuptools.command.build_py import build_py as _build_py from setuptools.command.build_py import build_py as _build_py
from setuptools.command.build_ext import build_ext as _build_ext from setuptools.command.build_ext import build_ext as _build_ext
from pkg_resources import working_set, EntryPoint
from distutils.errors import DistutilsExecError from distutils.errors import DistutilsExecError
from subprocess import Popen, PIPE from subprocess import Popen, PIPE
...@@ -112,8 +113,17 @@ class build_py(_build_py): ...@@ -112,8 +113,17 @@ class build_py(_build_py):
# run `make <target>` # run `make <target>`
def runmake(target): def runmake(target):
# NOTE we care to propagate setuptools path to subpython because it could
# be "inserted" to us by buildout. Propagating whole sys.path is more
# risky, as e.g. it can break gdb which is using python3, while
# building/testing under python2.
pypathv = [working_set.by_key['setuptools'].location]
pypath = os.environ.get('PYTHONPATH')
if pypath is not None:
pypathv.append(pypath)
err = os.system('make %s PYTHON="%s" PYTHONPATH="%s"' % \ err = os.system('make %s PYTHON="%s" PYTHONPATH="%s"' % \
(target, sys.executable, ':'.join(sys.path))) (target, sys.executable, ':'.join(pypathv)))
if err: if err:
raise DistutilsExecError('Failed to execute `make %s`' % target) raise DistutilsExecError('Failed to execute `make %s`' % target)
...@@ -144,7 +154,6 @@ class build_ext(_build_ext): ...@@ -144,7 +154,6 @@ class build_ext(_build_ext):
# register `func` as entry point `entryname` in `groupname` in distribution `distname` on the fly # register `func` as entry point `entryname` in `groupname` in distribution `distname` on the fly
def register_as_entrypoint(func, entryname, groupname, distname): def register_as_entrypoint(func, entryname, groupname, distname):
from pkg_resources import working_set, EntryPoint
dist = working_set.by_key[distname] dist = working_set.by_key[distname]
# register group if it is not yet registered # register group if it is not yet registered
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment