Before, the same file name was used for temporary download of several
patches. But download function uses a hard link for a patch using
whose schema is file:, thus download of the second patch broke the
original file of the first patch.

Commit 57f3dd73 (Write .installed.cfg only once, in safe way and only if
there's any change) changed the way how __buildout_installed__ is read
and then written back on part upgrade.

In particular, before it was always:

    on install:
    - get [] of installed parts from recipe.install()

    on update:
    - read __buildout_installed__
    - split it by \n

    on write back for both:
    - join by \n
    - write to __buildout_installed__

but after that patch on-update stopped doing that split by \n and on
writeback the code checks for installed_files to be string and does not
split it if it is.

Except the check does not work for unicode strings. Look e.g. what
currently happens with fluentd:

After first install it is

__buildout_installed__ =
!py!u'/srv/slapgrid/slappart6/srv/runner/software/73a7bf66fea06c703a70faaced67dabf/parts/fluentd\n/srv/slapgrid/slappart6/srv/runner/software/73a7bf66fea06c703a70faaced67dabf/bin/fluent-cat\n/srv/slapgrid/slappart6/srv/runner/software/73a7bf66fea06c703a70faaced67dabf/bin/fluent-gem\n/srv/slapgrid/slappart6/srv/runner/software/73a7bf66fea06c703a70faaced67dabf/bin/fluent-debug\n/srv/slapgrid/slappart6/srv/runner/software/73a7bf66fea06c703a70faaced67dabf/bin/fluentd\n/srv/slapgrid/slappart6/srv/runner/software/73a7bf66fea06c703a70faaced67dabf/bin/gem'

but after update it becomes

__buildout_installed__ =
!py!u'/\ns\nr\nv\n/\ns\nl\na\np\ng\nr\ni\nd\n/\ns\nl\na\np\np\na\nr\nt\n6\n/\ns\nr\nv\n/\nr\nu\nn\nn\ne\nr\n/\ns\no\nf\nt\nw\na\nr\ne\n/\n7\n3\na\n7\nb\nf\n6\n6\nf\ne\na\n0\n6\nc\n7\n0\n3\na\n7\n0\nf\na\na\nc\ne\nd\n6\n7\nd\na\nb\nf\n/\np\na\nr\nt\ns\n/\nf\nl\nu\ne\nn\nt\nd\n\n\n/\ns\nr\nv\n/\ns\nl\na\np\ng\nr\ni\nd\n/\ns\nl\na\np\np\na\nr\nt\n6\n/\ns\nr\nv\n/\nr\nu\nn\nn\ne\nr\n/\ns\no\nf\nt\nw\na\nr\ne\n/\n7\n3\na\n7\nb\nf\n6\n6\nf\ne\na\n0\n6\nc\n7\n0\n3\na\n7\n0\nf\na\na\nc\ne\nd\n6\n7\nd\na\nb\nf\n/\nb\ni\nn\n/\nf\nl\nu\ne\nn\nt\n-\nc\na\nt\n\n\n/\ns\nr\nv\n/\ns\nl\na\np\ng\nr\ni\nd\n/\ns\nl\na\np\np\na\nr\nt\n6\n/\ns\nr\nv\n/\nr\nu\nn\nn\ne\nr\n/\ns\no\nf\nt\nw\na\nr\ne\n/\n7\n3\na\n7\nb\nf\n6\n6\nf\ne\na\n0\n6\nc\n7\n0\n3\na\n7\n0\nf\na\na\nc\ne\nd\n6\n7\nd\na\nb\nf\n/\nb\ni\nn\n/\nf\nl\nu\ne\nn\nt\n-\ng\ne\nm\n\n\n/\ns\nr\nv\n/\ns\nl\na\np\ng\nr\ni\nd\n/\ns\nl\na\np\np\na\nr\nt\n6\n/\ns\nr\nv\n/\nr\nu\nn\nn\ne\nr\n/\ns\no\nf\nt\nw\na\nr\ne\n/\n7\n3\na\n7\nb\nf\n6\n6\nf\ne\na\n0\n6\nc\n7\n0\n3\na\n7\n0\nf\na\na\nc\ne\nd\n6\n7\nd\na\nb\nf\n/\nb\ni\nn\n/\nf\nl\nu\ne\nn\nt\n-\nd\ne\nb\nu\ng\n\n\n/\ns\nr\nv\n/\ns\nl\na\np\ng\nr\ni\nd\n/\ns\nl\na\np\np\na\nr\nt\n6\n/\ns\nr\nv\n/\nr\nu\nn\nn\ne\nr\n/\ns\no\nf\nt\nw\na\nr\ne\n/\n7\n3\na\n7\nb\nf\n6\n6\nf\ne\na\n0\n6\nc\n7\n0\n3\na\n7\n0\nf\na\na\nc\ne\nd\n6\n7\nd\na\nb\nf\n/\nb\ni\nn\n/\nf\nl\nu\ne\nn\nt\nd\n\n\n/\ns\nr\nv\n/\ns\nl\na\np\ng\nr\ni\nd\n/\ns\nl\na\np\np\na\nr\nt\n6\n/\ns\nr\nv\n/\nr\nu\nn\nn\ne\nr\n/\ns\no\nf\nt\nw\na\nr\ne\n/\n7\n3\na\n7\nb\nf\n6\n6\nf\ne\na\n0\n6\nc\n7\n0\n3\na\n7\n0\nf\na\na\nc\ne\nd\n6\n7\nd\na\nb\nf\n/\nb\ni\nn\n/\ng\ne\nm'

Correct the check to take unicode strings into account.

/cc @jm
/reviewed-by @kazuhiko

Previous attempt to fix possible corruption was reverted because it had severe
performance issues (both CPU & storage).

Also, updating a part does not put it anymore at the end of the list of
installed parts. It was breaking uninstall dependencies.

This reverts commit 54bbe6bf.

Conflicts:
	zc.recipe.egg_/setup.py
	zc.recipe.egg_/src/zc/recipe/egg/egg.py

Currently only zc.recipe.egg:custom supports setting environment
variables, and zc.recipe.egg:develop does not.

My motivation for allowing setting environment in :develop is
wendelin.core

    https://lab.nexedi.cn/nexedi/slapos/blob/b5faab3b/component/wendelin.core/buildout.cfg

There we have [wendelin.core] part which installs released egg from
pypi, and [wendelin.core-dev] part which installs wendelin.core from
its latest git version via zc.recipe.egg:develop .

The problem is, wendelin.core for setup.py to work, needs git available,
and with slapos we usually don't have git available on base system, so
we build it by our own and do something like

    [wendelin.core-dev]
    recipe = zc.recipe.egg:develop
    environment = wendelin.core-dev-env

    [wendelin.core-dev-env]
    # wendelin.core-dev needs git to build
    PATH = ${git:location}/bin:%(PATH)s

and the problem is environment does not currently work for
zc.recipe.egg:develop, and thus git is not found -> build fails.

~~~~

In order to support environment in :develop, we just move environment
setting/restoring bits from Custom to Base, and provide Base.install() which
uses this bits. Custom & Develop .install() becomes ._install() which gets
hooked into Base.install() .

I've tested the patch only manually, because currently automated tests are
broken in a lot of places for slapos.buildout and zc.recipe.egg .

/cc @kazuhiko, @Tyagov

- Support on the fly patches in zc.recipe.egg by ``EGGNAME-patches``,
  ``EGGNAME-patch-options``, ``EGGNAME-patch-binary`` (or
  ``patch-binary``) and ``EGGNAME-patch-revision`` options.
- Support on the fly patches in zc.recipe.egg:custom by ``patches``,
  ``patch-options``, ``patch-binary`` and ``patch-revision`` options.
  (options ``EGGNAME-*`` are also supported as well).

Merge branch 'always_use_build'

Merge branch 'include_postrelease_string_in_version'

This reduces the risk of leaving a corrupted .installed.cfg

Version string is now N.N.N.postN to follow http://legacy.python.org/dev/peps/pep-0440/ .
(Ideally it should be N.N.N+SlapOSNNN, but version string containing '+' sign does now work
with older setuptools, that can be still used at the very beginning of bootstrap process.)

Conflicts:
	.gitignore
	setup.py
	src/zc/buildout/buildout.py
	src/zc/buildout/download.py
	src/zc/buildout/easy_install.py
	src/zc/buildout/testing.py
	src/zc/buildout/tests.py

Add referred parts' hash strings in __buildout_signature__, that invokes rebuild of a part when one of its (recursive) dependencies are modified.

Also remove duplicates and sort entries in __buildout_signature__.