Fixup "[test] Add tests for build dependencies"

Fixup "[feat] zc.recipe.egg: Improve on the fly patches."

Adapt filter in tests for pip emitting Python2.7 deprecation warnings
to pip >= 20.2.2 and < 21.0 - these versions seem to accidentally add
a redundant bit of message to the warning.

Depending on the test environment, pip, setuptools and wheel dists
may be installed as .egg or .egg-link and may be files or folders.

Depending on the test environment, pip, setuptools and wheel dists
may be installed as .egg or .egg-link and may be files or folders.

The general case is that buildout will pick versions for setuptools,
pip and wheel when they are not pinned: it's only when these are
installed as .egg-link - either as actual develop eggs or as a
side-effect of being found in .dist-info format in the environment
e.g. in site-packages - that buildout does not consider it picked
a version for these eggs.

This affects tests that set `allow-picked-versions = false` or
`show-picked-versions = true` or `update-versions-file = ...`.

Fixup [fix] "Adapt to setuptools>=65.6.0 logging on root"

Fixup "[test] Add tests for build dependencies"

This creates a environment for tests that is isolated from the current
environment (as long as there are no .egg-link to e.g. site-packages).

The testing framework, some individual tests and 'buildout:develop'
call python setup.py <command> manually, which is deprecated by
setuptools. Depending on the setuptools version, suppress the
warning for now to avoid polluting the logs.

With recent setuptools, the way the test package index is laid out
causes a warning log to be emitted, which needs to be filtered out
to keep matching the expected test output.

Fixup "[fix] Fix working set sorting".

Some zc.recipe.egg tests were never adapted accordingly.

Fixup [feat] zc.recipe.egg: Improve on the fly patches.

Some zc.recipe.egg tests were never adapted to zc.buildout 3.0.1
now using pip and wheel in addition to setuptools.

Fixup [feat] Add dependencies in __buildout_signature__

Some zc.recipe.egg tests were never adapted to zc.buildout 3.0.1
now using pip and wheel in addition to setuptools.

Fixup "[test] Add tests for build dependencies".

That commit introduced new eggs in test package index and should
have modified the zc.recipe.egg tests accordingly also.

Fixup "[feat] zc.recipe.egg: Improve on the fly patches."

`zc.buildout.testing.easy_install_deprecated` should have been
`zc.buildout.testing.easyinstall_deprecated`.

With recent setuptools, calling `python setup.py sdist` to create an
archive for zc.recipe.egg creates a zc_recipe_egg-xyz.zip instead of
a zc.recipe.egg-xyz.zip. This is called during test setup to prepare
a local package index, as the tests prevent access to PyPI. The name
change leads the index to find an entry for zc-recipe-egg instead of
zc.recipe.egg (as normalization replaces _ with -), which causes the
tests to fail to find a distribution for zc.recipe.egg.

INFO logs from setuptools which were previously not emitted because
setuptool's ad-hoc legacy logger defaulted to WARNING and above may
now be emitted because setuptools now logs to the root logger, thus
the global root logger's level configuration applied.

This caused undesired 'root: <some setuptools info>' messages to be
emitted and caused many tests to fail due to unexpected outputs.

See merge request nexedi/slapos.buildout!30

When specifying an alternate URL as fallback, the main URL is always
used for both downloading & uploading from/to networkcache.

If slapos.libnetworkcache is importable, install it in bootstrap and
in buildout upgrade - the places where bin/buildout is (re)generated
- as though it were a dependency of zc.buildout.

This is a hack to propagate libnetworkcache as a soft dependency.

Like for URL that contain credentials, we still skip auth challenge
because it's faster and:
- we only support one auth scheme (basic)
- netrc provides no way to specify realms, which seem anyway to be
  less and less used (https://stackoverflow.com/q/69303610 reports
  that recent browsers don't display them anymore)

See merge request nexedi/slapos.buildout!25

This retries with the alternate URL in case of HTTPError with the main
one.

Used by slapos.recipe.build:download* and slapos.recipe.cmmi recipes.

- Py3: stop using legacy API of urllib.request and
       fix download of http(s) URLs containing user:passwd@
- Py2: avoid OOM when downloading huge files

This is implemented as a method in case we want to make it configurable
via [buildout].

An optimization is to avoid temporary file when possible: a rename
(or hard link) is not always possible (different mount points).

Another one is to not check md5sum twice when using cache file.

Fall-back mode is ignored if an MD5 checksum is given.

In case of checksum mismatch for a cached path, remove it and
download again, mainly to cover the following cases:
- the url content changes and the user updates the checksum
- buildout killed while downloading directly to cache
  (see above optimization)
- shutil.copyfile is interrupted

We want two identical sections at different URL to be able to produce
same signature. This feature is useful for slapos.recipe.cmmi's
shared parts.

This commit may fixup "Support ${:_profile_base_location_}." but is
purposefuly kept separate because it concerns parts signatures which
are an orthogonal feature.

When there is no pinned version for zc.buildout itself, buildout
adds a ">=<current-version>" requirement to prevent accidental
downgrading. If the current version has a local version label,
this produced an invalid version specifier. To fix this, only
the public part of the current version is used.