Write .installed.cfg only once, in safe way and only if there's any change.

Also, updating a part does not put it anymore at the end of the list of
installed parts, that was making .installed.cfg too big.

src/zc/buildout/debugging.txt

@@ -85,6 +85,8 @@ supply some input:
       File "/zc/buildout/buildout.py", line 1352, in main
         getattr(buildout, command)(args)
       File "/zc/buildout/buildout.py", line 383, in install
+        self._install_parts(install_args)
+      File buildout.py", line 791, in _install_parts
         installed_files = self[part]._call(recipe.install)
       File "/zc/buildout/buildout.py", line 961, in _call
         return f()

src/zc/buildout/tests.py

@@ -1498,7 +1498,7 @@ some evil recipes that exit uncleanly:
     >>> mkdir('recipes')
     >>> write('recipes', 'recipes.py',
     ... '''
-    ... import os
+    ... import sys
     ...
     ... class Clean:
     ...     def __init__(*_): pass
@@ -1506,10 +1506,10 @@ some evil recipes that exit uncleanly:
     ...     def update(_): pass
     ...
     ... class EvilInstall(Clean):
-    ...     def install(_): os._exit(1)
+    ...     def install(_): sys.exit(1)
     ...
     ... class EvilUpdate(Clean):
-    ...     def update(_): os._exit(1)
+    ...     def update(_): sys.exit(1)
     ... ''')
 
     >>> write('recipes', 'setup.py',
@@ -1606,7 +1606,6 @@ Now let's look at 3 cases:
     Uninstalling p2.
     Uninstalling p1.
     Uninstalling p4.
-    Uninstalling p3.
 
 3. We exit while installing or updating after uninstalling:
 
@@ -1682,7 +1681,6 @@ Now let's look at 3 cases:
 
     >>> print_(system(buildout), end='')
     Develop: '/sample-buildout/recipes'
-    Uninstalling p1.
     Installing p1.
     Updating p2.
     Updating p3.
@@ -2742,6 +2740,73 @@ def increment_on_command_line():
       recipe='zc.buildout:debug'
     """
 
+def bug_664539_simple_buildout():
+  r"""
+  >>> write('buildout.cfg', '''
+  ... [buildout]
+  ... parts = escape
+  ...
+  ... [escape]
+  ... recipe = zc.buildout:debug
+  ... foo = $${nonexistent:option}
+  ... ''')
+
+  >>> print_(system(buildout), end='')
+  Installing escape.
+    foo='${nonexistent:option}'
+    recipe='zc.buildout:debug'
+  """
+
+def bug_664539_reference():
+  r"""
+  >>> write('buildout.cfg', '''
+  ... [buildout]
+  ... parts = escape
+  ...
+  ... [escape]
+  ... recipe = zc.buildout:debug
+  ... foo = ${:bar}
+  ... bar = $${nonexistent:option}
+  ... ''')
+
+  >>> print_(system(buildout), end='')
+  Installing escape.
+    bar='${nonexistent:option}'
+    foo='${nonexistent:option}'
+    recipe='zc.buildout:debug'
+  """
+
+def bug_664539_complex_buildout():
+  r"""
+  >>> write('buildout.cfg', '''
+  ... [buildout]
+  ... parts = escape
+  ...
+  ... [escape]
+  ... recipe = zc.buildout:debug
+  ... foo = ${level1:foo}
+  ...
+  ... [level1]
+  ... recipe = zc.buildout:debug
+  ... foo = ${level2:foo}
+  ...
+  ... [level2]
+  ... recipe = zc.buildout:debug
+  ... foo = $${nonexistent:option}
+  ... ''')
+
+  >>> print_(system(buildout), end='')
+  Installing level2.
+    foo='${nonexistent:option}'
+    recipe='zc.buildout:debug'
+  Installing level1.
+    foo='${nonexistent:option}'
+    recipe='zc.buildout:debug'
+  Installing escape.
+    foo='${nonexistent:option}'
+    recipe='zc.buildout:debug'
+  """
+
 def test_constrained_requirement():
     """
     zc.buildout.easy_install._constrained_requirement(constraint, requirement)

[Julien Muchembled]

This squashes unrelated changes from the 1.x branch, and authorship is lost. The original commits should be cherry-picked again.

/cc @vpelletier

[Vincent Pelletier]

I confirm: I do not want commits cherry-picked from a public branch to be ever squashed, on any repository ever.

I did not try to confirm if this is a squashing commit, I trust @jm on this. I did not check the amount of work needed to set thehistory straight either, so I cannot comment on whether a re-cherry-pick (so an history rewrite ? or maybe a revert, then re-cherry-pick ?) is reasonable.

[Julien Muchembled]

Not sure what you mean actually. The 1.x branch is quite messy and for some bugfixes we have many try/revert commits. Also, I hope we push some of our changes upstream so keeping a clean patch queue is more important than preserving authorship.

But surely, when original commits are clean, they should be cherry-picked properly, in particular with -x option.

Things went wrong with this commit mainly because upstream did massive [dr]eindentation. A change that got lost is that 57f3dd73 stops putting updated part at the end of the list of installed parts, because this was breaking uninstall dependencies: in order to keep it, we'll also have to update unit tests.

In any case, there's still some work to do in this 2.x branch before using it in production.

/cc @rafael

[Kazuhiko Shiozaki]

In this commit, I avoided 'Indent big section of code before actual changes' commit that happend in 1.x branch and took a bit different approach to achieve the same goal. And the original file itself is quite different between 1.x and 2.x thus I did not cherry pick for this commit.

But indeed I also included non-related changes by mistake...

[Julien Muchembled]

Oops, I forgot it was me who did massive indentation. I'll have to reread all this carefully.

[Rafael Monnerat]

The 2.x series is already part of the latest slapos package, but for software releases they are only present on a few cases for now, like ERP5TestNode and webrunner (not the current production), otherwise nobody would be able to try it (or use it earlier).

There are few things which were fixed only on 2.5.2-nexedi branch, so probably few projects (like wendelin, woelfel..) may need to use because I don't think we are going to backport changes to 1.7.x anymore... but people can adopt as soon they few confortable with it.

Feel free to review 2.5.2-nexedi branch and work to contribute back the changes on mainstream, but on our side is better not wait much (we are waiting 6 months or more to have the chance to try).