Commit 3d6db516 authored by Kirill Smelkov's avatar Kirill Smelkov

Fix build for zodbtools/pygolang/nxdtest/... tests

Commit 7d1ea024 (version up: zc.buildout 2.7.1+slapos013) switched buildout to
not use egg wheels, even for wheels that are source-only without any binaries.
As the result some eggs stopped to install because they setup_require other
eggs and neither buildout handles setup_requires automatically, nor there is
explicit setup_require declaration in corresponding components.

An example of such egg is python-dateutil: it setup_requires setuptools-scm,
and without switching python-dateutil to zc.recipe.egg:custom with
setup_requires the build fails.

The way 7d1ea024 used to workaround this is to mark all such eggs with :whl
suffix marking them that "it is ok to use wheel for this particular egg". The
complete list of eggs for which such markings was done in 7d1ea024 is:

    ipykernel
    plantuml
    mock
    PasteScript
    backports.functools-lru-cache
    cliff
    stevedore
    CacheControl
    pytest-runner
    jsonschema
    lockfile
    python-dateutil

However this list is not really complete because after 7d1ea024 the build
started to fail for e.g. zodbtools test:

        Traceback (most recent call last):
          File "<string>", line 10, in <module>
          File "/srv/slapgrid/slappart16/srv/testnode/dfp/soft/c670f74c8de08964f63c4b9f7d48c3d1/eggs/setuptools-44.1.1-py2.7.egg/setuptools/command/easy_install.py", line 2321, in main
          ...
          File "/srv/slapgrid/slappart16/srv/testnode/dfp/soft/c670f74c8de08964f63c4b9f7d48c3d1/eggs/setuptools-44.1.1-py2.7.egg/pkg_resources/__init__.py", line 786, in resolve
            raise DistributionNotFound(req, requirers)
        pkg_resources.DistributionNotFound: The 'setuptools-scm' distribution was not found and is required by the application
        An error occurred when trying to install pytest 4.6.11. Look above this message for any errors that were output by easy_install.
        While:
          Installing zodbtools-python.
          Base installation request: 'zodbtools[test]'
            Requirement of zodbtools[test]: mock
            Requirement of zodbtools[test]: pytz
            Requirement of zodbtools[test]: freezegun
            Requirement of zodbtools[test]: pytest
            Requirement of zodbtools[test]: dateparser
            Requirement of zodbtools[test]: six
            Requirement of zodbtools[test]: pygolang>=0.0.0.dev6
            Requirement of zodbtools[test]: zope.interface
            Requirement of zodbtools[test]: zodburi
            Requirement of zodbtools[test]: ZODB
            Requirement of mock: funcsigs>=1
            Requirement of mock: six
            Requirement of freezegun: python-dateutil>=2.7
          Getting distribution for 'pytest==4.6.11'.
        Error: Couldn't install: pytest 4.6.11

and similarly for pygolang and nxdtest tests.

I would be more comfortable if the fix would be to teach buildout to either
handle setup_requires automatically, or to automatically install wheels if the
wheel is source-only. However, given the way - that 7d1ea024 started to use
to handle such situations - let's fix this via explicitly marking pytest, and
other related eggs that setup_require something and are source-only, as :whl.

NOTE we did not notice the problem earlier because on testnodes, after the
switch to new buildout, pytest egg was already installed and incremental
rebuild did not need to reinstall pytest egg. So most of pygolang and zodbtools
test results remained in "ok" state, and only zodbtools-kirr, that was recently
reconfigured or changed testnode, indicated this issue. That's why the problem
remained unnoticed for a long time.

/reviewed-by @jerome
/reviewed-on nexedi/slapos!1026
parent cc4b577d
...@@ -14,4 +14,4 @@ eggs = ${pytest:eggs} ...@@ -14,4 +14,4 @@ eggs = ${pytest:eggs}
[versions] [versions]
pytest = 4.6.11 pytest = 4.6.11:whl
...@@ -155,7 +155,7 @@ cliff = 2.8.3:whl ...@@ -155,7 +155,7 @@ cliff = 2.8.3:whl
cmd2 = 0.7.0 cmd2 = 0.7.0
collective.recipe.shelloutput = 0.1 collective.recipe.shelloutput = 0.1
collective.recipe.template = 2.0 collective.recipe.template = 2.0
configparser = 4.0.2 configparser = 4.0.2:whl
contextlib2 = 0.6.0.post1 contextlib2 = 0.6.0.post1
cryptography = 3.3.1 cryptography = 3.3.1
dateparser = 0.7.6 dateparser = 0.7.6
...@@ -165,7 +165,7 @@ gevent = 20.9.0 ...@@ -165,7 +165,7 @@ gevent = 20.9.0
geventmp = 0.0.1 geventmp = 0.0.1
greenlet = 0.4.17 greenlet = 0.4.17
idna = 2.9 idna = 2.9
importlib-metadata = 1.7.0 importlib-metadata = 1.7.0:whl
inotify-simple = 1.1.1 inotify-simple = 1.1.1
itsdangerous = 0.24 itsdangerous = 0.24
lock-file = 2.0 lock-file = 2.0
...@@ -178,8 +178,8 @@ pbr = 2.0.0 ...@@ -178,8 +178,8 @@ pbr = 2.0.0
plone.recipe.command = 1.1 plone.recipe.command = 1.1
prettytable = 0.7.2 prettytable = 0.7.2
psutil = 5.8.0 psutil = 5.8.0
pluggy = 0.13.1 pluggy = 0.13.1:whl
py = 1.9.0 py = 1.9.0:whl
pyOpenSSL = 19.1.0 pyOpenSSL = 19.1.0
pyparsing = 2.2.0 pyparsing = 2.2.0
pytz = 2016.10 pytz = 2016.10
...@@ -207,7 +207,7 @@ wheel = 0.35.1:whl ...@@ -207,7 +207,7 @@ wheel = 0.35.1:whl
xml-marshaller = 1.0.2 xml-marshaller = 1.0.2
zc.lockfile = 1.0.2 zc.lockfile = 1.0.2
zdaemon = 4.2.0 zdaemon = 4.2.0
zipp = 1.2.0 zipp = 1.2.0:whl
zodburi = 2.4.0 zodburi = 2.4.0
zope.event = 3.5.2 zope.event = 3.5.2
paramiko = 2.1.3 paramiko = 2.1.3
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment