component/ZODB: Fix test-zodb4-wc2.cfg build (2)

After previous patch the build of this test passes further than buildout rebootstrap but fails with

    Installing software release /srv/slapgrid/slappart5/srv/project/slapos/component/ZODB/test-zodb4-wc2.cfg ...
    ...
    Installing persistent-repository.
    Warning: "branch" parameter with value "master" is ignored. Checking out to revision 4.6.4-0-g7ed95cf.
    Cloning into '/srv/slapgrid/slappart5/srv/runner/software/04a9ddca848c1bb62254aab83f212066/parts/persistent'...
    HEAD is now at 7ed95cf Preparing release 4.6.4
    Updating python-cffi.
    Updating persistent.
    DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
    Installing BTrees.
    networkcache: Trying to download pypi:persistent=4.9.3 from network cache...
    Getting distribution for 'persistent==4.9.3'.
        ERROR: Command errored out with exit status 1:
         command: /srv/slapgrid/slappart5/srv/runner/shared/python2.7/474396ed0d332179d494b6d9d3f28ef5/bin/python2.7 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-kywY6K/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-kywY6K/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-LeIkZV
             cwd: /tmp/pip-req-build-kywY6K/
        Complete output (25 lines):
        DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
        WARNING: Url 'file:///dev/null/cffi/' is ignored: it is neither a file nor a directory.
        ERROR: Could not find a version that satisfies the requirement cffi (from versions: none)
        ERROR: No matching distribution found for cffi
        Traceback (most recent call last):
          File "<string>", line 1, in <module>
          File "/tmp/pip-req-build-kywY6K/setup.py", line 149, in <module>
            entry_points={})
          File "/srv/slapgrid/slappart5/srv/runner/software/04a9ddca848c1bb62254aab83f212066/eggs/setuptools-44.1.1-py2.7.egg/setuptools/__init__.py", line 161, in setup
            _install_setup_requires(attrs)
          File "/srv/slapgrid/slappart5/srv/runner/software/04a9ddca848c1bb62254aab83f212066/eggs/setuptools-44.1.1-py2.7.egg/setuptools/__init__.py", line 156, in _install_setup_requires
            dist.fetch_build_eggs(dist.setup_requires)
          File "/srv/slapgrid/slappart5/srv/runner/software/04a9ddca848c1bb62254aab83f212066/eggs/setuptools-44.1.1-py2.7.egg/setuptools/dist.py", line 721, in fetch_build_eggs
            replace_conflicting=True,
          File "/srv/slapgrid/slappart5/srv/runner/software/04a9ddca848c1bb62254aab83f212066/eggs/setuptools-44.1.1-py2.7.egg/pkg_resources/__init__.py", line 782, in resolve
            replace_conflicting=replace_conflicting
          File "/srv/slapgrid/slappart5/srv/runner/software/04a9ddca848c1bb62254aab83f212066/eggs/setuptools-44.1.1-py2.7.egg/pkg_resources/__init__.py", line 1065, in best_match
            return self.obtain(req, installer)
          File "/srv/slapgrid/slappart5/srv/runner/software/04a9ddca848c1bb62254aab83f212066/eggs/setuptools-44.1.1-py2.7.egg/pkg_resources/__init__.py", line 1077, in obtain
            return installer(requirement)
          File "/srv/slapgrid/slappart5/srv/runner/software/04a9ddca848c1bb62254aab83f212066/eggs/setuptools-44.1.1-py2.7.egg/setuptools/dist.py", line 777, in fetch_build_egg
            return fetch_build_egg(self, req)
          File "/srv/slapgrid/slappart5/srv/runner/software/04a9ddca848c1bb62254aab83f212066/eggs/setuptools-44.1.1-py2.7.egg/setuptools/installer.py", line 130, in fetch_build_egg
            raise DistutilsError(str(e))
        distutils.errors.DistutilsError: Command '['/srv/slapgrid/slappart5/srv/runner/shared/python2.7/474396ed0d332179d494b6d9d3f28ef5/bin/python2.7', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmptaQnzb', '--quiet', '--index-url', 'file:///dev/null', 'cffi']' returned non-zero exit status 1
        ----------------------------------------
    ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
    While:
      Installing BTrees.
      Base installation request: 'persistent'
      Getting distribution for 'persistent==4.9.3'.

    An internal error occurred due to a bug in either zc.buildout or in a
    recipe being used:
    Traceback (most recent call last):
      File "/srv/slapgrid/slappart5/srv/runner/software/04a9ddca848c1bb62254aab83f212066/eggs/zc.buildout-3.0.1+slapos009-py2.7.egg/zc/buildout/buildout.py", line 2664, in main
        getattr(buildout, command)(args)
      File "/srv/slapgrid/slappart5/srv/runner/software/04a9ddca848c1bb62254aab83f212066/eggs/zc.buildout-3.0.1+slapos009-py2.7.egg/zc/buildout/buildout.py", line 855, in install
        self._install_parts(install_args)
      File "/srv/slapgrid/slappart5/srv/runner/software/04a9ddca848c1bb62254aab83f212066/eggs/zc.buildout-3.0.1+slapos009-py2.7.egg/zc/buildout/buildout.py", line 1026, in _install_parts
        installed_files = self[part]._call(recipe.install)
      File "/srv/slapgrid/slappart5/srv/runner/software/04a9ddca848c1bb62254aab83f212066/eggs/zc.buildout-3.0.1+slapos009-py2.7.egg/zc/buildout/buildout.py", line 1948, in _call
        return f()
      File "/srv/slapgrid/slappart5/srv/runner/software/04a9ddca848c1bb62254aab83f212066/eggs/zc.recipe.egg-2.0.8.dev0+slapos009-py2.7.egg/zc/recipe/egg/custom.py", line 79, in install
        self._install_setup_eggs()
      File "/srv/slapgrid/slappart5/srv/runner/software/04a9ddca848c1bb62254aab83f212066/eggs/zc.recipe.egg-2.0.8.dev0+slapos009-py2.7.egg/zc/recipe/egg/custom.py", line 119, in _install_setup_eggs
        newest=self.newest,
      File "/srv/slapgrid/slappart5/srv/runner/software/04a9ddca848c1bb62254aab83f212066/eggs/zc.buildout-3.0.1+slapos009-py2.7.egg/zc/buildout/easy_install.py", line 1178, in install
        return installer.install(specs, working_set, patch_dict=patch_dict)
      File "/srv/slapgrid/slappart5/srv/runner/software/04a9ddca848c1bb62254aab83f212066/eggs/zc.buildout-3.0.1+slapos009-py2.7.egg/zc/buildout/easy_install.py", line 835, in install
        for dist in self._get_dist(requirement, ws):
      File "/srv/slapgrid/slappart5/srv/runner/software/04a9ddca848c1bb62254aab83f212066/eggs/zc.buildout-3.0.1+slapos009-py2.7.egg/zc/buildout/easy_install.py", line 711, in _get_dist
        dists = [_move_to_eggs_dir_and_compile(dist, self._dest, self)]
      File "/srv/slapgrid/slappart5/srv/runner/software/04a9ddca848c1bb62254aab83f212066/eggs/zc.buildout-3.0.1+slapos009-py2.7.egg/zc/buildout/easy_install.py", line 2205, in _move_to_eggs_dir_and_compile
        [tmp_loc] = call_pip_wheel(dist.location, tmp_dest, options)
      File "/srv/slapgrid/slappart5/srv/runner/software/04a9ddca848c1bb62254aab83f212066/eggs/zc.buildout-3.0.1+slapos009-py2.7.egg/zc/buildout/easy_install.py", line 2100, in call_pip_wheel
        call_pip_command(['wheel', '-w', dest], [spec], options)
      File "/srv/slapgrid/slappart5/srv/runner/software/04a9ddca848c1bb62254aab83f212066/eggs/zc.buildout-3.0.1+slapos009-py2.7.egg/zc/buildout/easy_install.py", line 2038, in call_pip_command
        subprocess.check_call(args, env=env)
      File "/srv/slapgrid/slappart5/srv/runner/shared/python2.7/474396ed0d332179d494b6d9d3f28ef5/lib/python2.7/subprocess.py", line 190, in check_call
        raise CalledProcessError(retcode, cmd)
    CalledProcessError: Command '['/srv/slapgrid/slappart5/srv/runner/shared/python2.7/474396ed0d332179d494b6d9d3f28ef5/bin/python2.7', '-m', 'pip', 'wheel', '-w', '/srv/slapgrid/slappart5/srv/runner/software/04a9ddca848c1bb62254aab83f212066/eggs/tmpmfO6r5', '--no-deps', '-q', '--no-index', '--no-build-isolation', '--no-python-version-warning', '--no-cache-dir', '/tmp/tmpbuYU1zget_dist/persistent-4.9.3.tar.gz']' returned non-zero exit status 1
    2025-03-03 06:25:33 slapos[1517575] ERROR Failed to run buildout profile in directory '/srv/slapgrid/slappart5/srv/runner/software/04a9ddca848c1bb62254aab83f212066'
    2025-03-03 06:25:33 slapos[1517575] INFO Finished software releases.

What happens here is that test-zodb4-wc2.cfg installs persistent via git clone
but at 4.6.4 version, while the version of wanted egg in ZODB/buildout.cfg
[versions] is 4.9.3. This way when buildout sees that BTrees needs persistent as
the dependency, it looks for "whether we already have persistent==4.9.3
installed", determines the answer as "no" (as we have persistent 4.6.4), and
tries to install persistent by itself, and fails somewhere around cffi which we
also provide via zc.recipe.egg:custom.

The cffi error is only a consequence of original problem of mismatched
persistent versions.

-> Fix BTrees/persistent build by keeping used persistent versions in sync in between buildout.cfg and test-zodb4-wc2.cfg

As test-zodb4-wc2.cfg explains

    we need persistent to be a git checkout because persistent tests want to
    discover in-tree files that are not present in persistent egg when it is
    installed in non-development mode:

      https://erp5.nexedi.net/test_result_module/20201123-3F859E35/7
      (look for "AssertionError: could not find my setup.py")

      https://github.com/zopefoundation/persistent/blob/4.6.4-0-g7ed95cf/persistent/tests/test_docs.py#L37-L43

and on the other hand we do not use git checkout way uniformly because people
wanted to use plain eggs when component/ZODB was bootstrapped.

This way we have to use both egg and git checkout and keep their versions in sync.

After this patch the build of test-zodb4-wc2 is recovered.

/cc @xavier_thompson
/reviewed-by @kazuhiko, @levin.zimmermann
/helped-and-reviewed-by @jerome
/reviewed-on !1752
10 jobs for master
latest
Status Job ID Name Coverage
  External
passed Cloudooo.UnitTest-Master

00:12:15

passed SlapOS.Eggs.UnitTest-Master.Python2

00:17:28

passed SlapOS.Eggs.UnitTest-Master.Python3

00:19:38

failed SlapOS.Eggs.UnitTest-Master.Python3Next

failed SlapOS.KVM.ResilienceTest-Master

00:47:25

running SlapOS.SoftwareReleases.IntegrationTest-Master

04:40:37

running SlapOS.Theia.ResilienceTest-Master-ERP5

04:39:49

running SlapOS.Theia.ResilienceTest-Master-Gitlab

04:39:21

failed Zodbtools.UnitTest-Master.ZODB6.Python3

00:00:48

failed SlapOS.Eggs.UnitTest-Master.Python3Next

00:30:21