In RawConfigParser:
  # ';' is a comment delimiter only if it follows
  # a spacing character
and this applies only on the first line of the value.

By escaping the first newline, which is by the way useless and ugly,
a ';' on the second line (i.e. the first continuation line) would be saved
on the first line in .installed.cfg.

This resulted in parts being reinstalled without reason.

WARNING: Newer versions of buildout don't use RawConfigParser anymore so it
         may not be affected by this bug and in any case, this commit must not
         be cherry-picked blindly.

That caused uninstall and install everytime for a part containing $$ in its
saved options. Also it caused broken .installed.cfg in case of error, like the
referenced section was not defined.

Before, the same file name was used for temporary download of several
patches. But download function uses a hard link for a patch using
whose schema is file:, thus download of the second patch broke the
original file of the first patch.

Commit 57f3dd73 (Write .installed.cfg only once, in safe way and only if
there's any change) changed the way how __buildout_installed__ is read
and then written back on part upgrade.

In particular, before it was always:

    on install:
    - get [] of installed parts from recipe.install()

    on update:
    - read __buildout_installed__
    - split it by \n

    on write back for both:
    - join by \n
    - write to __buildout_installed__

but after that patch on-update stopped doing that split by \n and on
writeback the code checks for installed_files to be string and does not
split it if it is.

Except the check does not work for unicode strings. Look e.g. what
currently happens with fluentd:

After first install it is

__buildout_installed__ =
!py!u'/srv/slapgrid/slappart6/srv/runner/software/73a7bf66fea06c703a70faaced67dabf/parts/fluentd\n/srv/slapgrid/slappart6/srv/runner/software/73a7bf66fea06c703a70faaced67dabf/bin/fluent-cat\n/srv/slapgrid/slappart6/srv/runner/software/73a7bf66fea06c703a70faaced67dabf/bin/fluent-gem\n/srv/slapgrid/slappart6/srv/runner/software/73a7bf66fea06c703a70faaced67dabf/bin/fluent-debug\n/srv/slapgrid/slappart6/srv/runner/software/73a7bf66fea06c703a70faaced67dabf/bin/fluentd\n/srv/slapgrid/slappart6/srv/runner/software/73a7bf66fea06c703a70faaced67dabf/bin/gem'

but after update it becomes

__buildout_installed__ =
!py!u'/\ns\nr\nv\n/\ns\nl\na\np\ng\nr\ni\nd\n/\ns\nl\na\np\np\na\nr\nt\n6\n/\ns\nr\nv\n/\nr\nu\nn\nn\ne\nr\n/\ns\no\nf\nt\nw\na\nr\ne\n/\n7\n3\na\n7\nb\nf\n6\n6\nf\ne\na\n0\n6\nc\n7\n0\n3\na\n7\n0\nf\na\na\nc\ne\nd\n6\n7\nd\na\nb\nf\n/\np\na\nr\nt\ns\n/\nf\nl\nu\ne\nn\nt\nd\n\n\n/\ns\nr\nv\n/\ns\nl\na\np\ng\nr\ni\nd\n/\ns\nl\na\np\np\na\nr\nt\n6\n/\ns\nr\nv\n/\nr\nu\nn\nn\ne\nr\n/\ns\no\nf\nt\nw\na\nr\ne\n/\n7\n3\na\n7\nb\nf\n6\n6\nf\ne\na\n0\n6\nc\n7\n0\n3\na\n7\n0\nf\na\na\nc\ne\nd\n6\n7\nd\na\nb\nf\n/\nb\ni\nn\n/\nf\nl\nu\ne\nn\nt\n-\nc\na\nt\n\n\n/\ns\nr\nv\n/\ns\nl\na\np\ng\nr\ni\nd\n/\ns\nl\na\np\np\na\nr\nt\n6\n/\ns\nr\nv\n/\nr\nu\nn\nn\ne\nr\n/\ns\no\nf\nt\nw\na\nr\ne\n/\n7\n3\na\n7\nb\nf\n6\n6\nf\ne\na\n0\n6\nc\n7\n0\n3\na\n7\n0\nf\na\na\nc\ne\nd\n6\n7\nd\na\nb\nf\n/\nb\ni\nn\n/\nf\nl\nu\ne\nn\nt\n-\ng\ne\nm\n\n\n/\ns\nr\nv\n/\ns\nl\na\np\ng\nr\ni\nd\n/\ns\nl\na\np\np\na\nr\nt\n6\n/\ns\nr\nv\n/\nr\nu\nn\nn\ne\nr\n/\ns\no\nf\nt\nw\na\nr\ne\n/\n7\n3\na\n7\nb\nf\n6\n6\nf\ne\na\n0\n6\nc\n7\n0\n3\na\n7\n0\nf\na\na\nc\ne\nd\n6\n7\nd\na\nb\nf\n/\nb\ni\nn\n/\nf\nl\nu\ne\nn\nt\n-\nd\ne\nb\nu\ng\n\n\n/\ns\nr\nv\n/\ns\nl\na\np\ng\nr\ni\nd\n/\ns\nl\na\np\np\na\nr\nt\n6\n/\ns\nr\nv\n/\nr\nu\nn\nn\ne\nr\n/\ns\no\nf\nt\nw\na\nr\ne\n/\n7\n3\na\n7\nb\nf\n6\n6\nf\ne\na\n0\n6\nc\n7\n0\n3\na\n7\n0\nf\na\na\nc\ne\nd\n6\n7\nd\na\nb\nf\n/\nb\ni\nn\n/\nf\nl\nu\ne\nn\nt\nd\n\n\n/\ns\nr\nv\n/\ns\nl\na\np\ng\nr\ni\nd\n/\ns\nl\na\np\np\na\nr\nt\n6\n/\ns\nr\nv\n/\nr\nu\nn\nn\ne\nr\n/\ns\no\nf\nt\nw\na\nr\ne\n/\n7\n3\na\n7\nb\nf\n6\n6\nf\ne\na\n0\n6\nc\n7\n0\n3\na\n7\n0\nf\na\na\nc\ne\nd\n6\n7\nd\na\nb\nf\n/\nb\ni\nn\n/\ng\ne\nm'

Correct the check to take unicode strings into account.

/cc @jm
/reviewed-by @kazuhiko

Previous attempt to fix possible corruption was reverted because it had severe
performance issues (both CPU & storage).

Also, updating a part does not put it anymore at the end of the list of
installed parts. It was breaking uninstall dependencies.

This reverts commit 54bbe6bf.

Conflicts:
	zc.recipe.egg_/setup.py
	zc.recipe.egg_/src/zc/recipe/egg/egg.py

Currently only zc.recipe.egg:custom supports setting environment
variables, and zc.recipe.egg:develop does not.

My motivation for allowing setting environment in :develop is
wendelin.core

    https://lab.nexedi.cn/nexedi/slapos/blob/b5faab3b/component/wendelin.core/buildout.cfg

There we have [wendelin.core] part which installs released egg from
pypi, and [wendelin.core-dev] part which installs wendelin.core from
its latest git version via zc.recipe.egg:develop .

The problem is, wendelin.core for setup.py to work, needs git available,
and with slapos we usually don't have git available on base system, so
we build it by our own and do something like

    [wendelin.core-dev]
    recipe = zc.recipe.egg:develop
    environment = wendelin.core-dev-env

    [wendelin.core-dev-env]
    # wendelin.core-dev needs git to build
    PATH = ${git:location}/bin:%(PATH)s

and the problem is environment does not currently work for
zc.recipe.egg:develop, and thus git is not found -> build fails.

~~~~

In order to support environment in :develop, we just move environment
setting/restoring bits from Custom to Base, and provide Base.install() which
uses this bits. Custom & Develop .install() becomes ._install() which gets
hooked into Base.install() .

I've tested the patch only manually, because currently automated tests are
broken in a lot of places for slapos.buildout and zc.recipe.egg .

/cc @kazuhiko, @Tyagov

- Support on the fly patches in zc.recipe.egg by ``EGGNAME-patches``,
  ``EGGNAME-patch-options``, ``EGGNAME-patch-binary`` (or
  ``patch-binary``) and ``EGGNAME-patch-revision`` options.
- Support on the fly patches in zc.recipe.egg:custom by ``patches``,
  ``patch-options``, ``patch-binary`` and ``patch-revision`` options.
  (options ``EGGNAME-*`` are also supported as well).

Merge branch 'always_use_build'

Merge branch 'include_postrelease_string_in_version'

This reduces the risk of leaving a corrupted .installed.cfg

Version string is now N.N.N.postN to follow http://legacy.python.org/dev/peps/pep-0440/ .
(Ideally it should be N.N.N+SlapOSNNN, but version string containing '+' sign does now work
with older setuptools, that can be still used at the very beginning of bootstrap process.)