Commit ed79f5ee authored by Gary Poster's avatar Gary Poster
parents f551ee56 af2511a0
Change History Change History
************** **************
1.5.0 (unreleased)
==================
New features:
- zc.buildout supports Python 2.7.
- By default, Buildout and the bootstrap script now prefer final versions of
Buildout, recipes, and extensions. This can be changed by using the
--accept-buildout-test-releases flag (or -t for short) when calling
bootstrap. This will hopefully allow beta releases of these items to
be more easily and safely made in the future.
NOTE: dependencies of your own software are not affected by this new
behavior. Buildout continues to choose the newest available versions
of your dependencies regardless of whether they are final releases. To
prevent this, use the pre-existing switch ``prefer-final = true`` in
the [buildout] section of your configuration file (see
http://pypi.python.org/pypi/zc.buildout#preferring-final-releases) or
pin your versions using a versions section (see
http://pypi.python.org/pypi/zc.buildout#repeatable-buildouts-controlling-eggs-used).
Bugs fixed:
- You can now again use virtualenv with Buildout. The new features to let
buildout be used with a system Python are disabled in this configuration,
and the previous script generation behavior (1.4.3) is used, even if
the new function ``zc.buildout.easy_install.sitepackage_safe_scripts``
is used.
1.5.0b2 (2010-04-29)
====================
This was a re-release of 1.4.3 in order to keep 1.5.0b1 release from hurting
workflows that combined virtualenv with zc.buildout.
1.5.0b1 (2010-04-29) 1.5.0b1 (2010-04-29)
==================== ====================
...@@ -10,8 +46,13 @@ New Features: ...@@ -10,8 +46,13 @@ New Features:
both from command line and from config files. (gotcha) both from command line and from config files. (gotcha)
- Buildout can be safely used with a system Python (or any Python with code - Buildout can be safely used with a system Python (or any Python with code
in site-packages), as long as you use the new z3c.recipe.scripts in site-packages), as long as you use (1) A fresh checkout, (2) the
recipe to generate scripts and interpreters, rather than zc.recipe.egg. new bootstrap.py, and (3) recipes that use the new
``zc.buildout.easy_install.sitepackage_safe_scripts`` function to generate
scripts and interpreters. Many recipes will need to be updated to use
this new function. The scripts and interpreters generated by
``zc.recipe.egg`` will continue to use the older function, not safe
with system Pythons. Use the ``z3c.recipe.scripts`` as a replacement.
zc.recipe.egg is still a fully supported, and simpler, way of zc.recipe.egg is still a fully supported, and simpler, way of
generating scripts and interpreters if you are using a "clean" Python, generating scripts and interpreters if you are using a "clean" Python,
...@@ -42,15 +83,15 @@ New Features: ...@@ -42,15 +83,15 @@ New Features:
* The buildout script generated by bootstrap honors more of the settings * The buildout script generated by bootstrap honors more of the settings
in the designated configuration file (e.g., buildout.cfg). in the designated configuration file (e.g., buildout.cfg).
* Correcly handle systems where pkg_resources is present but the rest of * Correctly handle systems where pkg_resources is present but the rest of
setuptools is missing (like Ubuntu installs). setuptools is missing (like Ubuntu installs).
https://bugs.launchpad.net/zc.buildout/+bug/410528 https://bugs.launchpad.net/zc.buildout/+bug/410528
- You can develop zc.buildout using Distribute instead of Setuptools. Use - You can develop zc.buildout using Distribute instead of Setuptools. Use
the --distribute option on the dev.py script. (Releases should be tested the --distribute option on the dev.py script. (Releases should be tested
with both Distribute and Setuptools.) The tests for zc.buildout pass with both Distribute and Setuptools.) The tests for zc.buildout pass
with Setuptools and Python 2.4, 2.5, and 2.6; and with Distribute and with Setuptools and Python 2.4, 2.5, 2.6, and 2.7; and with Distribute and
Python 2.5 and 2.6. Using zc.buildout with Distribute and Python 2.4 Python 2.5, 2.6, and 2.7. Using zc.buildout with Distribute and Python 2.4
is not recommended. is not recommended.
- The ``distribute-version`` now works in the [buildout] section, mirroring - The ``distribute-version`` now works in the [buildout] section, mirroring
......
include *.txt
recursive-include src *.txt
exclude MANIFEST.in buildout.cfg .bzrignore
System Python and zc.buildout 1.5
*********************************
The 1.5 line of zc.buildout introduced a number of changes.
Problems
========
As usual, please send questions and comments to the `distutils SIG
mailing list <mailto://distutils-sig@python.org>`_. Report bugs using
the `zc.buildout Launchpad Bug Tracker
<https://launchpad.net/zc.buildout/+bugs>`_.
If problems are keeping you from your work, here's an easy way to
revert to the old code temporarily: switch to a custom "emergency"
bootstrap script, available from
http://svn.zope.org/repos/main/zc.buildout/branches/1.4/bootstrap/bootstrap.py .
This customized script will select zc.buildout 1.4.4 by default.
zc.buildout 1.4.4 will not upgrade itself unless you explicitly specify
a new version. It will also prefer older versions of zc.recipe.egg and
some other common recipes. If you have trouble with other recipes,
consider using a standard buildout "versions" section to specify older
versions of these, as described in the Buildout documentation
(http://pypi.python.org/pypi/zc.buildout#repeatable-buildouts-controlling-eggs-used).
Working with a System Python
============================
While there are a number of new features available in zc.buildout 1.5,
the biggest is that Buildout itself supports usage with a system Python.
This can work if you follow a couple of simple rules.
1. Use the new bootstrap.py (available from
svn://svn.zope.org/repos/main/zc.buildout/trunk/bootstrap/bootstrap.py).
2. Use buildout recipes that have been upgraded to work with zc.buildout 1.5
and higher. Specifically, they should use
``zc.buildout.easy_install.sitepackage_safe_scripts`` to generate
their scripts, if any, rather than ``zc.buildout.easy_install.scripts``.
See the `Recipes That Support a System Python`_ section below for more
details on recipes that are available as of this writing, and
`Updating Recipes to Support a System Python`_ for instructions on
how to update a recipe. Note that you should generally only need to
update recipes that generate scripts.
It's important to note that recipes not upgraded for zc.buildout 1.5.0
should continue to work--just not with a system Python.
Using a system Python is inherently fragile. Using a clean,
freshly-installed Python without customization in site-packages is more
robust and repeatable. See some of the regression tests added to the
1.5.0 line for the kinds of issues that you can encounter with a system
Python, and see
http://pypi.python.org/pypi/z3c.recipe.scripts#including-site-packages-and-sitecustomize
for more discussion.
However, using a system Python can be very convenient, and the
zc.buildout code for this feature has been tested by many users already.
Moreover, it has automated tests to exercise the problems that have been
encountered and fixed.
Recipes That Support a System Python
====================================
zc.recipe.egg continues to generate old-style scripts that are not safe
for use with a system Python. This was done for backwards
compatibility, because it is integral to so many buildouts and used as a
dependency of so many other recipes.
If you want to generate new-style scripts that do support system Python
usage, use z3c.recipe.scripts instead
(http://pypi.python.org/pypi/z3c.recipe.scripts). z3c.recipe.scripts has
the same script and interpreter generation options as zc.recipe.egg,
plus a few more for the new features mentioned above. In the simplest
case, you should be able to simply change ``recipe = zc.recipe.egg`` to
``recipe = z3c.recipe.scripts`` in the pertinent sections of your
buildout configuration and your generated scripts will work with a system
Python.
This is the only updated recipe as of this writing. Others should be
updated soon: see their change documents for details, or see `Updating
Recipes to Support a System Python`_ for instructions on how to update
recipes yourself.
Templates for creating Python scripts with the z3c.recipe.filetemplate
recipe can be easily changed to support a system Python.
- If you don't care about supporting relative paths, simply using a
generated interpreter with the eggs you want should be sufficient, as
it was before. For instance, if the interpreter is named "py", use
``#!${buildout:bin-directory/py}`` or ``#!/usr/bin/env
${buildout:bin-directory/py}``).
- If you do care about relative paths, (``relative-paths = true`` in
your buildout configuration), then z3c.recipe.scripts does require a
bit more changes, as is usual for the relative path support in that
package. First, use z3c.recipe.scripts to generate a script or
interpreter with the dependencies you want. This will create a
directory in ``parts`` that has a site.py and sitecustomize.py. Then,
begin your script as in the snippet below. The example assumes that
the z3c.recipe.scripts generated were from a Buildout configuration
section labeled "scripts": adjust accordingly.
::
#!${buildout:executable} -S
${python-relative-path-setup}
import sys
sys.path.insert(0, ${scripts:parts-directory|path-repr})
import site
Updating Recipes to Support a System Python
===========================================
You should generally only need to update recipes that generate scripts.
These recipes need to change from using ``zc.buildout.easy_install.scripts``
to be using ``zc.buildout.easy_install.sitepackage_safe_scripts``.
The signatures of the two functions are different. Please compare::
def scripts(
reqs, working_set, executable, dest,
scripts=None,
extra_paths=(),
arguments='',
interpreter=None,
initialization='',
relative_paths=False,
):
def sitepackage_safe_scripts(
dest, working_set, executable, site_py_dest,
reqs=(),
scripts=None,
interpreter=None,
extra_paths=(),
initialization='',
include_site_packages=False,
exec_sitecustomize=False,
relative_paths=False,
script_arguments='',
script_initialization='',
):
In most cases, the arguments are merely reordered. The ``reqs``
argument is no longer required in order to make it easier to generate an
interpreter alone. The ``arguments`` argument was renamed to
``script_arguments`` to clarify that it did not affect interpreter
generation.
The only new required argument is ``site_py_dest``. It must be the path
to a directory in which the customized site.py and sitecustomize.py
files will be written. A typical generation in a recipe will look like
this.
(In the recipe's __init__ method...)
::
self.options = options
b_options = buildout['buildout']
options['parts-directory'] = os.path.join(
b_options['parts-directory'], self.name)
(In the recipe's install method...)
::
options = self.options
generated = []
if not os.path.exists(options['parts-directory']):
os.mkdir(options['parts-directory'])
generated.append(options['parts-directory'])
Then ``options['parts-directory']`` can be used for the ``site_py_dest``
value.
If you want to support the other arguments (``include_site_packages``,
``exec_sitecustomize``, ``script_initialization``, as well as the
``allowed-eggs-from-site-packages`` option), you might want to look at
some of the code in
svn://svn.zope.org/repos/main/zc.buildout/trunk/z3c.recipe.scripts\_/src/z3c/recipe/scripts/scripts.py .
You might even be able to adopt some of it by subclassing or delegating.
The Scripts class in that file is the closest to what you might be used
to from zc.recipe.egg.
Important note for recipe authors: the code in recipes is *always run
without access to the site-packages*. This is irrespective of the
``include-site-packages`` option discussed elsewhere, which controls the
software being built, but not the environment in which Buildout itself runs.
virtualenv
==========
Using virtualenv (http://pypi.python.org/pypi/virtualenv) with the
--no-site-packages option already provided a simple way of using a
system Python. This is intended to continue to work, and some automated
tests exist to demonstrate this.
However, it is only supported to the degree that people have found it to
work in the past. The existing Buildout tests for virtualenv are only
for problems encountered previously. They are very far from
comprehensive.
Using Buildout with a system python has at least three advantages over
using Buildout in conjunction with virtualenv. They may or may not be
pertinent to your desired usage.
- Unlike ``virtualenv --no-site-packages``, Buildout's support allows you
to choose to let packages from your system Python be available to your
software (see ``include-site-packages`` in
http://pypi.python.org/pypi/z3c.recipe.scripts).
You can even specify which eggs installed in your system Python can be
allowed to fulfill some of your packages' dependencies (see
``allowed-eggs-from-site-packages`` in
http://pypi.python.org/pypi/z3c.recipe.scripts).
At the expense of some repeatability and platform dependency, this
flexibility means that, for instance, you can rely on
difficult-to-build eggs like lxml coming from your system Python.
- Buildout's implementation has a full set of automated tests.
- An integral Buildout implementation means fewer steps and fewer dependencies
to work with a system Python.
...@@ -18,99 +18,241 @@ The script accepts buildout command-line options, so you can ...@@ -18,99 +18,241 @@ The script accepts buildout command-line options, so you can
use the -c option to specify an alternate configuration file. use the -c option to specify an alternate configuration file.
""" """
import os, shutil, sys, tempfile, urllib2 import os, shutil, sys, tempfile, textwrap, urllib, urllib2, subprocess
from optparse import OptionParser from optparse import OptionParser
tmpeggs = tempfile.mkdtemp() if sys.platform == 'win32':
def quote(c):
if ' ' in c:
return '"%s"' % c # work around spawn lamosity on windows
else:
return c
else:
quote = str
# See zc.buildout.easy_install._has_broken_dash_S for motivation and comments.
stdout, stderr = subprocess.Popen(
[sys.executable, '-Sc',
'try:\n'
' import ConfigParser\n'
'except ImportError:\n'
' print 1\n'
'else:\n'
' print 0\n'],
stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()
has_broken_dash_S = bool(int(stdout.strip()))
# In order to be more robust in the face of system Pythons, we want to
# run without site-packages loaded. This is somewhat tricky, in
# particular because Python 2.6's distutils imports site, so starting
# with the -S flag is not sufficient. However, we'll start with that:
if not has_broken_dash_S and 'site' in sys.modules:
# We will restart with python -S.
args = sys.argv[:]
args[0:0] = [sys.executable, '-S']
args = map(quote, args)
os.execv(sys.executable, args)
# Now we are running with -S. We'll get the clean sys.path, import site
# because distutils will do it later, and then reset the path and clean
# out any namespace packages from site-packages that might have been
# loaded by .pth files.
clean_path = sys.path[:]
import site
sys.path[:] = clean_path
for k, v in sys.modules.items():
if (hasattr(v, '__path__') and
len(v.__path__)==1 and
not os.path.exists(os.path.join(v.__path__[0],'__init__.py'))):
# This is a namespace package. Remove it.
sys.modules.pop(k)
is_jython = sys.platform.startswith('java') is_jython = sys.platform.startswith('java')
setuptools_source = 'http://peak.telecommunity.com/dist/ez_setup.py'
distribute_source = 'http://python-distribute.org/distribute_setup.py'
# parsing arguments # parsing arguments
parser = OptionParser() def normalize_to_url(option, opt_str, value, parser):
if value:
if '://' not in value: # It doesn't smell like a URL.
value = 'file://%s' % (
urllib.pathname2url(
os.path.abspath(os.path.expanduser(value))),)
if opt_str == '--download-base' and not value.endswith('/'):
# Download base needs a trailing slash to make the world happy.
value += '/'
else:
value = None
name = opt_str[2:].replace('-', '_')
setattr(parser.values, name, value)
usage = '''\
[DESIRED PYTHON FOR BUILDOUT] bootstrap.py [options]
Bootstraps a buildout-based project.
Simply run this script in a directory containing a buildout.cfg, using the
Python that you want bin/buildout to use.
Note that by using --setup-source and --download-base to point to
local resources, you can keep this script from going over the network.
'''
parser = OptionParser(usage=usage)
parser.add_option("-v", "--version", dest="version", parser.add_option("-v", "--version", dest="version",
help="use a specific zc.buildout version") help="use a specific zc.buildout version")
parser.add_option("-d", "--distribute", parser.add_option("-d", "--distribute",
action="store_true", dest="distribute", default=False, action="store_true", dest="use_distribute", default=False,
help="Use Distribute rather than Setuptools.") help="Use Distribute rather than Setuptools.")
parser.add_option("--setup-source", action="callback", dest="setup_source",
callback=normalize_to_url, nargs=1, type="string",
help=("Specify a URL or file location for the setup file. "
"If you use Setuptools, this will default to " +
setuptools_source + "; if you use Distribute, this "
"will default to " + distribute_source +"."))
parser.add_option("--download-base", action="callback", dest="download_base",
callback=normalize_to_url, nargs=1, type="string",
help=("Specify a URL or directory for downloading "
"zc.buildout and either Setuptools or Distribute. "
"Defaults to PyPI."))
parser.add_option("--eggs",
help=("Specify a directory for storing eggs. Defaults to "
"a temporary directory that is deleted when the "
"bootstrap script completes."))
parser.add_option("-t", "--accept-buildout-test-releases",
dest='accept_buildout_test_releases',
action="store_true", default=False,
help=("Normally, if you do not specify a --version, the "
"bootstrap script and buildout gets the newest "
"*final* versions of zc.buildout and its recipes and "
"extensions for you. If you use this flag, "
"bootstrap and buildout will get the newest releases "
"even if they are alphas or betas."))
parser.add_option("-c", None, action="store", dest="config_file", parser.add_option("-c", None, action="store", dest="config_file",
help=("Specify the path to the buildout configuration " help=("Specify the path to the buildout configuration "
"file to be used.")) "file to be used."))
options, args = parser.parse_args() options, args = parser.parse_args()
# if -c was provided, we push it back into args for buildout' main function # if -c was provided, we push it back into args for buildout's main function
if options.config_file is not None: if options.config_file is not None:
args += ['-c', options.config_file] args += ['-c', options.config_file]
if options.version is not None: if options.eggs:
VERSION = '==%s' % options.version eggs_dir = os.path.abspath(os.path.expanduser(options.eggs))
else: else:
VERSION = '' eggs_dir = tempfile.mkdtemp()
if options.setup_source is None:
if options.use_distribute:
options.setup_source = distribute_source
else:
options.setup_source = setuptools_source
USE_DISTRIBUTE = options.distribute if options.accept_buildout_test_releases:
args = args + ['bootstrap'] args.append('buildout:accept-buildout-test-releases=true')
args.append('bootstrap')
try: try:
import pkg_resources import pkg_resources
import setuptools import setuptools # A flag. Sometimes pkg_resources is installed alone.
if not hasattr(pkg_resources, '_distribute'): if not hasattr(pkg_resources, '_distribute'):
raise ImportError raise ImportError
except ImportError: except ImportError:
ez_code = urllib2.urlopen(
options.setup_source).read().replace('\r\n', '\n')
ez = {} ez = {}
if USE_DISTRIBUTE: exec ez_code in ez
exec urllib2.urlopen('http://python-distribute.org/distribute_setup.py' setup_args = dict(to_dir=eggs_dir, download_delay=0)
).read() in ez if options.download_base:
ez['use_setuptools'](to_dir=tmpeggs, download_delay=0, no_fake=True) setup_args['download_base'] = options.download_base
else: if options.use_distribute:
exec urllib2.urlopen('http://peak.telecommunity.com/dist/ez_setup.py' setup_args['no_fake'] = True
).read() in ez ez['use_setuptools'](**setup_args)
ez['use_setuptools'](to_dir=tmpeggs, download_delay=0)
reload(sys.modules['pkg_resources']) reload(sys.modules['pkg_resources'])
import pkg_resources import pkg_resources
# This does not (always?) update the default working set. We will
# do it.
for path in sys.path:
if path not in pkg_resources.working_set.entries:
pkg_resources.working_set.add_entry(path)
if sys.platform == 'win32': cmd = [quote(sys.executable),
def quote(c): '-c',
if ' ' in c: quote('from setuptools.command.easy_install import main; main()'),
return '"%s"' % c # work around spawn lamosity on windows '-mqNxd',
else: quote(eggs_dir)]
return c
else:
def quote (c):
return c
cmd = 'from setuptools.command.easy_install import main; main()' if not has_broken_dash_S:
ws = pkg_resources.working_set cmd.insert(1, '-S')
if USE_DISTRIBUTE: find_links = options.download_base
requirement = 'distribute' if not find_links:
find_links = os.environ.get('bootstrap-testing-find-links')
if find_links:
cmd.extend(['-f', quote(find_links)])
if options.use_distribute:
setup_requirement = 'distribute'
else: else:
requirement = 'setuptools' setup_requirement = 'setuptools'
ws = pkg_resources.working_set
setup_requirement_path = ws.find(
pkg_resources.Requirement.parse(setup_requirement)).location
env = dict(
os.environ,
PYTHONPATH=setup_requirement_path)
requirement = 'zc.buildout'
version = options.version
if version is None and not options.accept_buildout_test_releases:
# Figure out the most recent final version of zc.buildout.
import setuptools.package_index
_final_parts = '*final-', '*final'
def _final_version(parsed_version):
for part in parsed_version:
if (part[:1] == '*') and (part not in _final_parts):
return False
return True
index = setuptools.package_index.PackageIndex(
search_path=[setup_requirement_path])
if find_links:
index.add_find_links((find_links,))
req = pkg_resources.Requirement.parse(requirement)
if index.obtain(req) is not None:
best = []
bestv = None
for dist in index[req.project_name]:
distv = dist.parsed_version
if _final_version(distv):
if bestv is None or distv > bestv:
best = [dist]
bestv = distv
elif distv == bestv:
best.append(dist)
if best:
best.sort()
version = best[-1].version
if version:
requirement = '=='.join((requirement, version))
cmd.append(requirement)
if is_jython: if is_jython:
import subprocess import subprocess
exitcode = subprocess.Popen(cmd, env=env).wait()
else: # Windows prefers this, apparently; otherwise we would prefer subprocess
exitcode = os.spawnle(*([os.P_WAIT, sys.executable] + cmd + [env]))
if exitcode != 0:
sys.stdout.flush()
sys.stderr.flush()
print ("An error occurred when trying to install zc.buildout. "
"Look above this message for any errors that "
"were output by easy_install.")
sys.exit(exitcode)
assert subprocess.Popen([sys.executable] + ['-c', quote(cmd), '-mqNxd', ws.add_entry(eggs_dir)
quote(tmpeggs), 'zc.buildout' + VERSION], ws.require(requirement)
env=dict(os.environ,
PYTHONPATH=
ws.find(pkg_resources.Requirement.parse(requirement)).location
),
).wait() == 0
else:
assert os.spawnle(
os.P_WAIT, sys.executable, quote (sys.executable),
'-c', quote (cmd), '-mqNxd', quote (tmpeggs), 'zc.buildout' + VERSION,
dict(os.environ,
PYTHONPATH=
ws.find(pkg_resources.Requirement.parse(requirement)).location
),
) == 0
ws.add_entry(tmpeggs)
ws.require('zc.buildout' + VERSION)
import zc.buildout.buildout import zc.buildout.buildout
zc.buildout.buildout.main(args) zc.buildout.buildout.main(args)
shutil.rmtree(tmpeggs) if not options.eggs: # clean up temporary egg directory
shutil.rmtree(eggs_dir)
##############################################################################
#
# Copyright (c) 2006 Zope Foundation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
##############################################################################
"""Bootstrap a buildout-based project
Simply run this script in a directory containing a buildout.cfg.
The script accepts buildout command-line options, so you can
use the -c option to specify an alternate configuration file.
"""
import os, shutil, sys, tempfile, textwrap, urllib, urllib2
from optparse import OptionParser
if sys.platform == 'win32':
def quote(c):
if ' ' in c:
return '"%s"' % c # work around spawn lamosity on windows
else:
return c
else:
quote = str
# In order to be more robust in the face of system Pythons, we want to
# run without site-packages loaded. This is somewhat tricky, in
# particular because Python 2.6's distutils imports site, so starting
# with the -S flag is not sufficient. However, we'll start with that:
if 'site' in sys.modules:
# We will restart with python -S.
args = sys.argv[:]
args[0:0] = [sys.executable, '-S']
args = map(quote, args)
os.execv(sys.executable, args)
# Now we are running with -S. We'll get the clean sys.path, import site
# because distutils will do it later, and then reset the path and clean
# out any namespace packages from site-packages that might have been
# loaded by .pth files.
clean_path = sys.path[:]
import site
sys.path[:] = clean_path
for k, v in sys.modules.items():
if (hasattr(v, '__path__') and
len(v.__path__)==1 and
not os.path.exists(os.path.join(v.__path__[0],'__init__.py'))):
# This is a namespace package. Remove it.
sys.modules.pop(k)
is_jython = sys.platform.startswith('java')
setuptools_source = 'http://peak.telecommunity.com/dist/ez_setup.py'
distribute_source = 'http://python-distribute.org/distribute_setup.py'
# parsing arguments
def normalize_to_url(option, opt_str, value, parser):
if value:
if '://' not in value: # It doesn't smell like a URL.
value = 'file://%s' % (
urllib.pathname2url(
os.path.abspath(os.path.expanduser(value))),)
if opt_str == '--download-base' and not value.endswith('/'):
# Download base needs a trailing slash to make the world happy.
value += '/'
else:
value = None
name = opt_str[2:].replace('-', '_')
setattr(parser.values, name, value)
usage = '''\
[DESIRED PYTHON FOR BUILDOUT] bootstrap.py [options]
Bootstraps a buildout-based project.
Simply run this script in a directory containing a buildout.cfg, using the
Python that you want bin/buildout to use.
Note that by using --setup-source and --download-base to point to
local resources, you can keep this script from going over the network.
'''
parser = OptionParser(usage=usage)
parser.add_option("-v", "--version", dest="version",
help="use a specific zc.buildout version")
parser.add_option("-d", "--distribute",
action="store_true", dest="use_distribute", default=False,
help="Use Distribute rather than Setuptools.")
parser.add_option("--setup-source", action="callback", dest="setup_source",
callback=normalize_to_url, nargs=1, type="string",
help=("Specify a URL or file location for the setup file. "
"If you use Setuptools, this will default to " +
setuptools_source + "; if you use Distribute, this "
"will default to " + distribute_source +"."))
parser.add_option("--download-base", action="callback", dest="download_base",
callback=normalize_to_url, nargs=1, type="string",
help=("Specify a URL or directory for downloading "
"zc.buildout and either Setuptools or Distribute. "
"Defaults to PyPI."))
parser.add_option("--eggs",
help=("Specify a directory for storing eggs. Defaults to "
"a temporary directory that is deleted when the "
"bootstrap script completes."))
parser.add_option("-c", None, action="store", dest="config_file",
help=("Specify the path to the buildout configuration "
"file to be used."))
options, args = parser.parse_args()
# if -c was provided, we push it back into args for buildout's main function
if options.config_file is not None:
args += ['-c', options.config_file]
if options.eggs:
eggs_dir = os.path.abspath(os.path.expanduser(options.eggs))
else:
eggs_dir = tempfile.mkdtemp()
if options.setup_source is None:
if options.use_distribute:
options.setup_source = distribute_source
else:
options.setup_source = setuptools_source
args = args + ['bootstrap']
try:
import pkg_resources
import setuptools # A flag. Sometimes pkg_resources is installed alone.
if not hasattr(pkg_resources, '_distribute'):
raise ImportError
except ImportError:
ez_code = urllib2.urlopen(
options.setup_source).read().replace('\r\n', '\n')
ez = {}
exec ez_code in ez
setup_args = dict(to_dir=eggs_dir, download_delay=0)
if options.download_base:
setup_args['download_base'] = options.download_base
if options.use_distribute:
setup_args['no_fake'] = True
ez['use_setuptools'](**setup_args)
reload(sys.modules['pkg_resources'])
import pkg_resources
# This does not (always?) update the default working set. We will
# do it.
for path in sys.path:
if path not in pkg_resources.working_set.entries:
pkg_resources.working_set.add_entry(path)
cmd = [quote(sys.executable),
'-c',
quote('from setuptools.command.easy_install import main; main()'),
'-mqNxd',
quote(eggs_dir)]
if options.download_base:
cmd.extend(['-f', quote(options.download_base)])
requirement = 'zc.buildout'
if options.version:
requirement = '=='.join((requirement, options.version))
cmd.append(requirement)
if options.use_distribute:
setup_requirement = 'distribute'
else:
setup_requirement = 'setuptools'
ws = pkg_resources.working_set
env = dict(
os.environ,
PYTHONPATH=ws.find(
pkg_resources.Requirement.parse(setup_requirement)).location)
if is_jython:
import subprocess
exitcode = subprocess.Popen(cmd, env=env).wait()
else: # Windows prefers this, apparently; otherwise we would prefer subprocess
exitcode = os.spawnle(*([os.P_WAIT, sys.executable] + cmd + [env]))
if exitcode != 0:
sys.stdout.flush()
sys.stderr.flush()
print ("An error occurred when trying to install zc.buildout. "
"Look above this message for any errors that "
"were output by easy_install.")
sys.exit(exitcode)
ws.add_entry(eggs_dir)
ws.require(requirement)
import zc.buildout.buildout
zc.buildout.buildout.main(args)
if not options.eggs: # clean up temporary egg directory
shutil.rmtree(eggs_dir)
...@@ -11,16 +11,15 @@ interpreter = py ...@@ -11,16 +11,15 @@ interpreter = py
[test] [test]
recipe = zc.recipe.testrunner recipe = zc.recipe.testrunner
eggs = eggs =
zc.buildout zc.buildout[test]
zc.recipe.egg zc.recipe.egg
z3c.recipe.scripts z3c.recipe.scripts
zope.testing
# Tests that can be run wo a network # Tests that can be run wo a network
[oltest] [oltest]
recipe = zc.recipe.testrunner recipe = zc.recipe.testrunner
eggs = eggs =
zc.buildout zc.buildout[test]
zc.recipe.egg zc.recipe.egg
z3c.recipe.scripts z3c.recipe.scripts
defaults = defaults =
......
...@@ -19,7 +19,7 @@ buildout egg itself is installed as a develop egg. ...@@ -19,7 +19,7 @@ buildout egg itself is installed as a develop egg.
$Id$ $Id$
""" """
import os, shutil, sys, subprocess, urllib2 import os, shutil, sys, subprocess, urllib2, subprocess
from optparse import OptionParser from optparse import OptionParser
if sys.platform == 'win32': if sys.platform == 'win32':
...@@ -31,11 +31,15 @@ if sys.platform == 'win32': ...@@ -31,11 +31,15 @@ if sys.platform == 'win32':
else: else:
quote = str quote = str
# Detect https://bugs.launchpad.net/virtualenv/+bug/572545 .
has_broken_dash_S = subprocess.call(
[sys.executable, '-Sc', 'import ConfigParser'])
# In order to be more robust in the face of system Pythons, we want to # In order to be more robust in the face of system Pythons, we want to
# run without site-packages loaded. This is somewhat tricky, in # run without site-packages loaded. This is somewhat tricky, in
# particular because Python 2.6's distutils imports site, so starting # particular because Python 2.6's distutils imports site, so starting
# with the -S flag is not sufficient. However, we'll start with that: # with the -S flag is not sufficient. However, we'll start with that:
if 'site' in sys.modules: if not has_broken_dash_S and 'site' in sys.modules:
# We will restart with python -S. # We will restart with python -S.
args = sys.argv[:] args = sys.argv[:]
args[0:0] = [sys.executable, '-S'] args[0:0] = [sys.executable, '-S']
...@@ -117,10 +121,14 @@ except ImportError: ...@@ -117,10 +121,14 @@ except ImportError:
env = os.environ.copy() # Windows needs yet-to-be-determined values from this. env = os.environ.copy() # Windows needs yet-to-be-determined values from this.
env['PYTHONPATH'] = os.path.dirname(pkg_resources.__file__) env['PYTHONPATH'] = os.path.dirname(pkg_resources.__file__)
subprocess.Popen(
[sys.executable] + cmd = [sys.executable,
['setup.py', '-q', 'develop', '-m', '-x', '-d', 'develop-eggs'], 'setup.py', '-q', 'develop', '-m', '-x', '-d', 'develop-eggs']
env=env).wait()
if not has_broken_dash_S:
cmd.insert(1, '-S')
subprocess.Popen(cmd, env=env).wait()
pkg_resources.working_set.add_entry('src') pkg_resources.working_set.add_entry('src')
......
...@@ -12,7 +12,7 @@ ...@@ -12,7 +12,7 @@
# #
############################################################################## ##############################################################################
name = "zc.buildout" name = "zc.buildout"
version = "1.5.0dev" version = "1.5.0"
import os import os
from setuptools import setup from setuptools import setup
...@@ -23,6 +23,8 @@ def read(*rnames): ...@@ -23,6 +23,8 @@ def read(*rnames):
long_description=( long_description=(
read('README.txt') read('README.txt')
+ '\n' + + '\n' +
read('SYSTEM_PYTHON_HELP.txt')
+ '\n' +
'Detailed Documentation\n' 'Detailed Documentation\n'
'**********************\n' '**********************\n'
+ '\n' + + '\n' +
...@@ -83,6 +85,7 @@ setup( ...@@ -83,6 +85,7 @@ setup(
install_requires = 'setuptools', install_requires = 'setuptools',
include_package_data = True, include_package_data = True,
entry_points = entry_points, entry_points = entry_points,
extras_require = dict(test=['zope.testing']),
zip_safe=False, zip_safe=False,
classifiers = [ classifiers = [
'Intended Audience :: Developers', 'Intended Audience :: Developers',
......
...@@ -2,14 +2,14 @@ Allow hosts ...@@ -2,14 +2,14 @@ Allow hosts
----------- -----------
On some environments the links visited by `zc.buildout` can be forbidden On some environments the links visited by `zc.buildout` can be forbidden
by paranoiac firewalls. These URL might be on the chain of links by paranoiac firewalls. These URL might be on the chain of links
visited by `zc.buildout` whether they are defined in the `find-links` option visited by `zc.buildout` whether they are defined in the `find-links` option
or by various eggs in their `url`, `download_url` and `dependency_links` metadata. or by various eggs in their `url`, `download_url` and `dependency_links` metadata.
It is even harder to track that package_index works like a spider and It is even harder to track that package_index works like a spider and
might visit links and go to other location. might visit links and go to other location.
The `allow-hosts` option provides a way to prevent this, and The `allow-hosts` option provides a way to prevent this, and
works exactly like the one provided in `easy_install` works exactly like the one provided in `easy_install`
(see `easy_install allow-hosts option`_). (see `easy_install allow-hosts option`_).
...@@ -17,7 +17,7 @@ You can provide a list of allowed host, together with wildcards:: ...@@ -17,7 +17,7 @@ You can provide a list of allowed host, together with wildcards::
[buildout] [buildout]
... ...
allow-hosts = allow-hosts =
*.python.org *.python.org
example.com example.com
...@@ -36,7 +36,7 @@ Let's create a develop egg in our buildout that specifies ...@@ -36,7 +36,7 @@ Let's create a develop egg in our buildout that specifies
... zip_safe=True, version='1') ... zip_safe=True, version='1')
... ''') ... ''')
Now let's configure the buildout to use the develop egg, Now let's configure the buildout to use the develop egg,
together with some rules that disallow any website but PyPI and together with some rules that disallow any website but PyPI and
local files:: local files::
...@@ -55,14 +55,11 @@ local files:: ...@@ -55,14 +55,11 @@ local files::
Now we can run the buildout and make sure all attempts to dist.plone.org fails:: Now we can run the buildout and make sure all attempts to dist.plone.org fails::
>>> print system(buildout) >>> print system(buildout) # doctest: +ELLIPSIS
Develop: '/sample-buildout/allowdemo' Develop: '/sample-buildout/allowdemo'
Installing eggs. ...
<BLANKLINE>
Link to http://dist.plone.org ***BLOCKED*** by --allow-hosts Link to http://dist.plone.org ***BLOCKED*** by --allow-hosts
<BLANKLINE> ...
Couldn't find index page for 'kss.core' (maybe misspelled?)
Getting distribution for 'kss.core'.
While: While:
Installing eggs. Installing eggs.
Getting distribution for 'kss.core'. Getting distribution for 'kss.core'.
...@@ -91,14 +88,11 @@ XXX (showcase with a svn:// file) ...@@ -91,14 +88,11 @@ XXX (showcase with a svn:// file)
Now we can run the buildout and make sure all attempts to dist.plone.org fails:: Now we can run the buildout and make sure all attempts to dist.plone.org fails::
>>> print system(buildout) >>> print system(buildout) # doctest: +ELLIPSIS
Develop: '/sample-buildout/allowdemo' Develop: '/sample-buildout/allowdemo'
Installing eggs. ...
<BLANKLINE>
Link to http://dist.plone.org ***BLOCKED*** by --allow-hosts Link to http://dist.plone.org ***BLOCKED*** by --allow-hosts
<BLANKLINE> ...
Couldn't find index page for 'kss.core' (maybe misspelled?)
Getting distribution for 'kss.core'.
While: While:
Installing eggs. Installing eggs.
Getting distribution for 'kss.core'. Getting distribution for 'kss.core'.
...@@ -115,7 +109,7 @@ Test for 1.0.5 breakage as in https://bugs.launchpad.net/zc.buildout/+bug/239212 ...@@ -115,7 +109,7 @@ Test for 1.0.5 breakage as in https://bugs.launchpad.net/zc.buildout/+bug/239212
... [buildout] ... [buildout]
... parts=python ... parts=python
... foo = ${python:interpreter} ... foo = ${python:interpreter}
... ...
... [python] ... [python]
... recipe=zc.recipe.egg ... recipe=zc.recipe.egg
... eggs=zc.buildout ... eggs=zc.buildout
......
...@@ -47,16 +47,116 @@ Make sure the bootstrap script actually works:: ...@@ -47,16 +47,116 @@ Make sure the bootstrap script actually works::
X... X...
d zc.buildout-...egg d zc.buildout-...egg
The buildout script it has generated is a new-style script, using a
customized site.py.
>>> buildout_script = join(sample_buildout, 'bin', 'buildout')
>>> if sys.platform.startswith('win'):
... buildout_script += '-script.py'
>>> print open(buildout_script).read() # doctest: +ELLIPSIS
#...
<BLANKLINE>
import sys
sys.path[0:0] = [
'/sample/parts/buildout',
]
<BLANKLINE>
<BLANKLINE>
import os
path = sys.path[0]
if os.environ.get('PYTHONPATH'):
path = os.pathsep.join([path, os.environ['PYTHONPATH']])
os.environ['PYTHONPATH'] = path
import site # imports custom buildout-generated site.py
<BLANKLINE>
import zc.buildout.buildout
<BLANKLINE>
if __name__ == '__main__':
zc.buildout.buildout.main()
<BLANKLINE>
The bootstrap process prefers final versions of zc.buildout, so it has
selected the (generated-locally) 99.99 egg rather than the also-available
100.0b1 egg. We can see that in the buildout script's site.py.
>>> buildout_site_py = join(
... sample_buildout, 'parts', 'buildout', 'site.py')
>>> print open(buildout_site_py).read() # doctest: +ELLIPSIS
"...
buildout_paths = [
'/sample/eggs/setuptools-...egg',
'/sample/eggs/zc.buildout-99.99-pyN.N.egg'
]
...
If you want to accept early releases of zc.buildout, you either need to
specify an explicit version (using --version here and specifying the
version in the buildout configuration file using the
``buildout-version`` option or the ``versions`` option) or specify that you
accept early releases by using ``--accept-buildout-test-releases`` on the
bootstrap script.
Here's an example.
>>> ignored = system(
... zc.buildout.easy_install._safe_arg(sys.executable)+' '+
... 'bootstrap.py --accept-buildout-test-releases')
>>> print open(buildout_site_py).read() # doctest: +ELLIPSIS
"...
buildout_paths = [
'/sample/eggs/setuptools-...egg',
'/sample/eggs/zc.buildout-100.0b1-pyN.N.egg'
]
...
Notice we are now using zc.buildout 100.0b1, a non-final release.
The buildout script remembers the decision to accept early releases, and
alerts the user.
>>> print system(join('bin', 'buildout')),
... # doctest: +NORMALIZE_WHITESPACE
NOTE: Accepting early releases of build system packages. Rerun bootstrap
without --accept-buildout-test-releases (-t) to return to default
behavior.
This is accomplished within the script itself.
>>> print open(buildout_script).read() # doctest: +ELLIPSIS
#...
sys.argv.insert(1, 'buildout:accept-buildout-test-releases=true')
print ('NOTE: Accepting early releases of build system packages. Rerun '
'bootstrap without --accept-buildout-test-releases (-t) to return to '
'default behavior.')
...
As the note says, to undo, you just need to re-run bootstrap without
--accept-buildout-test-releases.
>>> ignored = system(
... zc.buildout.easy_install._safe_arg(sys.executable)+' '+
... 'bootstrap.py')
>>> print open(buildout_site_py).read() # doctest: +ELLIPSIS
"...
buildout_paths = [
'/sample/eggs/setuptools-...egg',
'/sample/eggs/zc.buildout-99.99-pyN.N.egg'
]
...
>>> ('buildout:accept-buildout-test-releases=true' in
... open(buildout_script).read())
False
Now we will try the `--version` option, which lets you define a version for Now we will try the `--version` option, which lets you define a version for
`zc.buildout`. If not provided, bootstrap will look for the latest one. `zc.buildout`. If not provided, bootstrap will look for the latest one.
Let's try with an unknown version:: Let's try with an unknown version::
>>> print 'X'; print system( >>> print 'XX'; print system(
... zc.buildout.easy_install._safe_arg(sys.executable)+' '+ ... zc.buildout.easy_install._safe_arg(sys.executable)+' '+
... 'bootstrap.py --version UNKNOWN'); print 'X' # doctest: +ELLIPSIS ... 'bootstrap.py --version UNKNOWN'); print 'X' # doctest: +ELLIPSIS
... ...
X X...
No local packages or download links found for zc.buildout==UNKNOWN... No local packages or download links found for zc.buildout==UNKNOWN...
... ...
...@@ -71,11 +171,9 @@ Now let's try with `1.1.2`, which happens to exist:: ...@@ -71,11 +171,9 @@ Now let's try with `1.1.2`, which happens to exist::
<BLANKLINE> <BLANKLINE>
X X
Let's make sure the generated `buildout` script uses it:: Versions older than 1.5.0 put their egg dependencies in the ``buildout`` script.
Let's make sure it was generated as we expect::
>>> buildout_script = join(sample_buildout, 'bin', 'buildout')
>>> if sys.platform.startswith('win'):
... buildout_script += '-script.py'
>>> print open(buildout_script).read() # doctest: +ELLIPSIS >>> print open(buildout_script).read() # doctest: +ELLIPSIS
#... #...
<BLANKLINE> <BLANKLINE>
...@@ -102,7 +200,7 @@ Let's try with `1.2.1`:: ...@@ -102,7 +200,7 @@ Let's try with `1.2.1`::
<BLANKLINE> <BLANKLINE>
X X
Let's make sure the generated `buildout` script uses it:: Let's make sure the generated ``buildout`` script uses it::
>>> print open(buildout_script).read() # doctest: +ELLIPSIS >>> print open(buildout_script).read() # doctest: +ELLIPSIS
#... #...
...@@ -119,7 +217,7 @@ Let's make sure the generated `buildout` script uses it:: ...@@ -119,7 +217,7 @@ Let's make sure the generated `buildout` script uses it::
zc.buildout.buildout.main() zc.buildout.buildout.main()
<BLANKLINE> <BLANKLINE>
`zc.buildout` now can also run with `Distribute` with the `--distribute` ``zc.buildout`` now can also run with `Distribute` with the `--distribute`
option:: option::
>>> print 'X'; print system( >>> print 'X'; print system(
...@@ -128,26 +226,17 @@ option:: ...@@ -128,26 +226,17 @@ option::
... ...
X X
... ...
Generated script '/sample/bin/buildout'. Generated script '/sample/bin/buildout'...
<BLANKLINE>
X X
Let's make sure the generated `buildout` script uses it:: Let's make sure the generated ``site.py`` uses it::
>>> print open(buildout_site_py).read() # doctest: +ELLIPSIS
>>> print open(buildout_script).read() # doctest: +ELLIPSIS "...
#... buildout_paths = [
<BLANKLINE> '/sample/eggs/distribute-...egg',
import sys '/sample/eggs/zc.buildout-99.99-pyN.N.egg'
sys.path[0:0] = [ ]
'/sample/eggs/distribute-...egg', ...
'/sample/eggs/zc.buildout-...egg',
]
<BLANKLINE>
import zc.buildout.buildout
<BLANKLINE>
if __name__ == '__main__':
zc.buildout.buildout.main()
<BLANKLINE>
Make sure both options can be used together:: Make sure both options can be used together::
...@@ -158,12 +247,11 @@ Make sure both options can be used together:: ...@@ -158,12 +247,11 @@ Make sure both options can be used together::
... ...
X X
... ...
Generated script '/sample/bin/buildout'. Generated script '/sample/bin/buildout'...
<BLANKLINE>
X X
Let's make sure the generated `buildout` script uses ``Distribute`` *and* Let's make sure the old-style generated ``buildout`` script uses
``zc.buildout-1.2.1``:: ``Distribute`` *and* ``zc.buildout-1.2.1``::
>>> print open(buildout_script).read() # doctest: +ELLIPSIS >>> print open(buildout_script).read() # doctest: +ELLIPSIS
#... #...
...@@ -192,22 +280,80 @@ Last, the -c option needs to work on bootstrap.py:: ...@@ -192,22 +280,80 @@ Last, the -c option needs to work on bootstrap.py::
... ...
X X
... ...
Generated script '/sample/bin/buildout'. Generated script '/sample/bin/buildout'...
<BLANKLINE>
X X
You can specify a location of ez_setup.py or distribute_setup, so you
can rely on a local or remote location. We'll write our own ez_setup.py
that we will also use to test some other bootstrap options.
>>> write('ez_setup.py', '''\
... def use_setuptools(**kwargs):
... import sys, pprint
... pprint.pprint(kwargs, width=40)
... sys.exit()
... ''')
>>> print system(
... zc.buildout.easy_install._safe_arg(sys.executable)+' '+
... 'bootstrap.py --setup-source=./ez_setup.py')
... # doctest: +ELLIPSIS
{'download_delay': 0,
'to_dir': '...'}
<BLANKLINE>
You can also pass a download-cache, and a place in which eggs should be stored
(they are normally stored in a temporary directory).
>>> print system(
... zc.buildout.easy_install._safe_arg(sys.executable)+' '+
... 'bootstrap.py --setup-source=./ez_setup.py '+
... '--download-base=./download-cache --eggs=eggs')
... # doctest: +ELLIPSIS
{'download_base': '/sample/download-cache/',
'download_delay': 0,
'to_dir': '/sample/eggs'}
<BLANKLINE>
Here's the entire help text. Here's the entire help text.
>>> print system( >>> print system(
... zc.buildout.easy_install._safe_arg(sys.executable)+' '+ ... zc.buildout.easy_install._safe_arg(sys.executable)+' '+
... 'bootstrap.py --help'), ... 'bootstrap.py --help'),
... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
Usage: bootstrap.py [options] Usage: [DESIRED PYTHON FOR BUILDOUT] bootstrap.py [options]
<BLANKLINE>
Bootstraps a buildout-based project.
<BLANKLINE>
Simply run this script in a directory containing a buildout.cfg, using the
Python that you want bin/buildout to use.
<BLANKLINE>
Note that by using --setup-source and --download-base to point to
local resources, you can keep this script from going over the network.
<BLANKLINE>
<BLANKLINE> <BLANKLINE>
Options: Options:
-h, --help show this help message and exit -h, --help show this help message and exit
-v VERSION, --version=VERSION -v VERSION, --version=VERSION
use a specific zc.buildout version use a specific zc.buildout version
-d, --distribute Use Distribute rather than Setuptools. -d, --distribute Use Distribute rather than Setuptools.
--setup-source=SETUP_SOURCE
Specify a URL or file location for the setup file. If
you use Setuptools, this will default to
http://peak.telecommunity.com/dist/ez_setup.py; if you
use Distribute, this will default to http://python-
distribute.org/distribute_setup.py.
--download-base=DOWNLOAD_BASE
Specify a URL or directory for downloading zc.buildout
and either Setuptools or Distribute. Defaults to PyPI.
--eggs=EGGS Specify a directory for storing eggs. Defaults to a
temporary directory that is deleted when the bootstrap
script completes.
-t, --accept-buildout-test-releases
Normally, if you do not specify a --version, the
bootstrap script and buildout gets the newest *final*
versions of zc.buildout and its recipes and extensions
for you. If you use this flag, bootstrap and buildout
will get the newest releases even if they are alphas
or betas.
-c CONFIG_FILE Specify the path to the buildout configuration file to -c CONFIG_FILE Specify the path to the buildout configuration file to
be used. be used.
...@@ -34,6 +34,7 @@ import shutil ...@@ -34,6 +34,7 @@ import shutil
import sys import sys
import tempfile import tempfile
import UserDict import UserDict
import warnings
import zc.buildout import zc.buildout
import zc.buildout.download import zc.buildout.download
import zc.buildout.easy_install import zc.buildout.easy_install
...@@ -51,6 +52,9 @@ is_jython = sys.platform.startswith('java') ...@@ -51,6 +52,9 @@ is_jython = sys.platform.startswith('java')
if is_jython: if is_jython:
import subprocess import subprocess
_sys_executable_has_broken_dash_S = (
zc.buildout.easy_install._has_broken_dash_S(sys.executable))
class MissingOption(zc.buildout.UserError, KeyError): class MissingOption(zc.buildout.UserError, KeyError):
"""A required option was missing. """A required option was missing.
""" """
...@@ -112,6 +116,7 @@ def _unannotate(data): ...@@ -112,6 +116,7 @@ def _unannotate(data):
return data return data
_buildout_default_options = _annotate_section({ _buildout_default_options = _annotate_section({
'accept-buildout-test-releases': 'false',
'allow-hosts': '*', 'allow-hosts': '*',
'allow-picked-versions': 'true', 'allow-picked-versions': 'true',
'bin-directory': 'bin', 'bin-directory': 'bin',
...@@ -230,6 +235,8 @@ class Buildout(UserDict.DictMixin): ...@@ -230,6 +235,8 @@ class Buildout(UserDict.DictMixin):
self._logger = logging.getLogger('zc.buildout') self._logger = logging.getLogger('zc.buildout')
self.offline = (buildout_section['offline'] == 'true') self.offline = (buildout_section['offline'] == 'true')
self.newest = (buildout_section['newest'] == 'true') self.newest = (buildout_section['newest'] == 'true')
self.accept_buildout_test_releases = (
buildout_section['accept-buildout-test-releases'] == 'true')
################################################################## ##################################################################
## WARNING!!! ## WARNING!!!
...@@ -263,42 +270,26 @@ class Buildout(UserDict.DictMixin): ...@@ -263,42 +270,26 @@ class Buildout(UserDict.DictMixin):
self._setup_logging() self._setup_logging()
offline = options['offline']
if offline not in ('true', 'false'):
self._error('Invalid value for offline option: %s', offline)
self.offline = (offline == 'true')
if self.offline:
newest = options['newest'] = 'false'
else:
newest = options['newest']
if newest not in ('true', 'false'):
self._error('Invalid value for newest option: %s', newest)
self.newest = (newest == 'true')
versions = options.get('versions') versions = options.get('versions')
if versions: if versions:
zc.buildout.easy_install.default_versions(dict(self[versions])) zc.buildout.easy_install.default_versions(dict(self[versions]))
prefer_final = options['prefer-final']
if prefer_final not in ('true', 'false'):
self._error('Invalid value for prefer-final option: %s',
prefer_final)
zc.buildout.easy_install.prefer_final(prefer_final=='true')
use_dependency_links = options['use-dependency-links'] self.offline = options.get_bool('offline')
if use_dependency_links not in ('true', 'false'): if self.offline:
self._error('Invalid value for use-dependency-links option: %s', options['newest'] = 'false'
use_dependency_links) self.newest = options.get_bool('newest')
zc.buildout.easy_install.prefer_final(
options.get_bool('prefer-final'))
self.accept_buildout_test_releases = options.get_bool(
'accept-buildout-test-releases')
zc.buildout.easy_install.use_dependency_links( zc.buildout.easy_install.use_dependency_links(
use_dependency_links == 'true') options.get_bool('use-dependency-links'))
allow_picked_versions = options['allow-picked-versions']
if allow_picked_versions not in ('true', 'false'):
self._error('Invalid value for allow-picked-versions option: %s',
allow_picked_versions)
zc.buildout.easy_install.allow_picked_versions( zc.buildout.easy_install.allow_picked_versions(
allow_picked_versions == 'true') options.get_bool('allow-picked-versions'))
zc.buildout.easy_install.install_from_cache(
options.get_bool('install-from-cache'))
zc.buildout.easy_install.always_unzip(options.get_bool('unzip'))
download_cache = options.get('download-cache') download_cache = options.get('download-cache')
if download_cache: if download_cache:
...@@ -315,19 +306,6 @@ class Buildout(UserDict.DictMixin): ...@@ -315,19 +306,6 @@ class Buildout(UserDict.DictMixin):
zc.buildout.easy_install.download_cache(download_cache) zc.buildout.easy_install.download_cache(download_cache)
install_from_cache = options['install-from-cache']
if install_from_cache not in ('true', 'false'):
self._error('Invalid value for install-from-cache option: %s',
install_from_cache)
zc.buildout.easy_install.install_from_cache(
install_from_cache=='true')
always_unzip = options['unzip']
if always_unzip not in ('true', 'false'):
self._error('Invalid value for unzip option: %s',
always_unzip)
zc.buildout.easy_install.always_unzip(always_unzip=='true')
# "Use" each of the defaults so they aren't reported as unused options. # "Use" each of the defaults so they aren't reported as unused options.
for name in _buildout_default_options: for name in _buildout_default_options:
options[name] options[name]
...@@ -359,7 +337,8 @@ class Buildout(UserDict.DictMixin): ...@@ -359,7 +337,8 @@ class Buildout(UserDict.DictMixin):
distributions, options['executable'], distributions, options['executable'],
[options['develop-eggs-directory'], [options['develop-eggs-directory'],
options['eggs-directory']], options['eggs-directory']],
include_site_packages=False, include_site_packages=_sys_executable_has_broken_dash_S,
prefer_final=not self.accept_buildout_test_releases,
) )
else: else:
ws = zc.buildout.easy_install.install( ws = zc.buildout.easy_install.install(
...@@ -370,7 +349,8 @@ class Buildout(UserDict.DictMixin): ...@@ -370,7 +349,8 @@ class Buildout(UserDict.DictMixin):
path=[options['develop-eggs-directory']], path=[options['develop-eggs-directory']],
newest=self.newest, newest=self.newest,
allow_hosts=self._allow_hosts, allow_hosts=self._allow_hosts,
include_site_packages=False, include_site_packages=_sys_executable_has_broken_dash_S,
prefer_final=not self.accept_buildout_test_releases,
) )
# Now copy buildout and setuptools eggs, and record destination eggs: # Now copy buildout and setuptools eggs, and record destination eggs:
...@@ -393,7 +373,9 @@ class Buildout(UserDict.DictMixin): ...@@ -393,7 +373,9 @@ class Buildout(UserDict.DictMixin):
else: else:
shutil.copy2(dist.location, dest) shutil.copy2(dist.location, dest)
# Create buildout script # Create buildout script.
# Ideally the (possibly) new version of buildout would get a
# chance to write the script. Not sure how to do that.
ws = pkg_resources.WorkingSet(entries) ws = pkg_resources.WorkingSet(entries)
ws.require('zc.buildout') ws.require('zc.buildout')
partsdir = os.path.join(options['parts-directory'], 'buildout') partsdir = os.path.join(options['parts-directory'], 'buildout')
...@@ -406,9 +388,19 @@ class Buildout(UserDict.DictMixin): ...@@ -406,9 +388,19 @@ class Buildout(UserDict.DictMixin):
else: else:
assert relative_paths == 'false' assert relative_paths == 'false'
relative_paths = '' relative_paths = ''
if (self.accept_buildout_test_releases and
self._annotated['buildout']['accept-buildout-test-releases'][1] ==
'COMMAND_LINE_VALUE'):
# Bootstrap was called with '--accept-buildout-test-releases'.
# Continue to honor that setting.
script_initialization = _early_release_initialization_code
else:
script_initialization = ''
zc.buildout.easy_install.sitepackage_safe_scripts( zc.buildout.easy_install.sitepackage_safe_scripts(
options['bin-directory'], ws, options['executable'], partsdir, options['bin-directory'], ws, options['executable'], partsdir,
reqs=['zc.buildout'], relative_paths=relative_paths) reqs=['zc.buildout'], relative_paths=relative_paths,
include_site_packages=_sys_executable_has_broken_dash_S,
script_initialization=script_initialization,)
init = bootstrap init = bootstrap
...@@ -422,7 +414,7 @@ class Buildout(UserDict.DictMixin): ...@@ -422,7 +414,7 @@ class Buildout(UserDict.DictMixin):
# for eggs: # for eggs:
sys.path.insert(0, self['buildout']['develop-eggs-directory']) sys.path.insert(0, self['buildout']['develop-eggs-directory'])
# Check for updates. This could cause the process to be rstarted # Check for updates. This could cause the process to be restarted.
self._maybe_upgrade() self._maybe_upgrade()
# load installed data # load installed data
...@@ -475,7 +467,7 @@ class Buildout(UserDict.DictMixin): ...@@ -475,7 +467,7 @@ class Buildout(UserDict.DictMixin):
# compute new part recipe signatures # compute new part recipe signatures
self._compute_part_signatures(install_parts) self._compute_part_signatures(install_parts)
# uninstall parts that are no-longer used or who's configs # uninstall parts that are no-longer used or whose configs
# have changed # have changed
for part in reversed(installed_parts): for part in reversed(installed_parts):
if part in install_parts: if part in install_parts:
...@@ -621,11 +613,11 @@ class Buildout(UserDict.DictMixin): ...@@ -621,11 +613,11 @@ class Buildout(UserDict.DictMixin):
f.close() f.close()
def _uninstall_part(self, part, installed_part_options): def _uninstall_part(self, part, installed_part_options):
# ununstall part # uninstall part
__doing__ = 'Uninstalling %s.', part __doing__ = 'Uninstalling %s.', part
self._logger.info(*__doing__) self._logger.info(*__doing__)
# run uinstall recipe # run uninstall recipe
recipe, entry = _recipe(installed_part_options[part]) recipe, entry = _recipe(installed_part_options[part])
try: try:
uninstaller = _install_and_load( uninstaller = _install_and_load(
...@@ -854,7 +846,8 @@ class Buildout(UserDict.DictMixin): ...@@ -854,7 +846,8 @@ class Buildout(UserDict.DictMixin):
index = options.get('index'), index = options.get('index'),
path = [options['develop-eggs-directory']], path = [options['develop-eggs-directory']],
allow_hosts = self._allow_hosts, allow_hosts = self._allow_hosts,
include_site_packages=False include_site_packages=_sys_executable_has_broken_dash_S,
prefer_final=not self.accept_buildout_test_releases,
) )
upgraded = [] upgraded = []
...@@ -902,15 +895,27 @@ class Buildout(UserDict.DictMixin): ...@@ -902,15 +895,27 @@ class Buildout(UserDict.DictMixin):
# the new dist is different, so we've upgraded. # the new dist is different, so we've upgraded.
# Update the scripts and return True # Update the scripts and return True
# Ideally the new version of buildout would get a chance to write the
# script. Not sure how to do that.
partsdir = os.path.join(options['parts-directory'], 'buildout') partsdir = os.path.join(options['parts-directory'], 'buildout')
if os.path.exists(partsdir): if os.path.exists(partsdir):
# This is primarily for unit tests, in which .py files change too # This is primarily for unit tests, in which .py files change too
# fast for Python to know to regenerate the .pyc/.pyo files. # fast for Python to know to regenerate the .pyc/.pyo files.
shutil.rmtree(partsdir) shutil.rmtree(partsdir)
os.mkdir(partsdir) os.mkdir(partsdir)
if (self.accept_buildout_test_releases and
self._annotated['buildout']['accept-buildout-test-releases'][1] ==
'COMMAND_LINE_VALUE'):
# Bootstrap was called with '--accept-buildout-test-releases'.
# Continue to honor that setting.
script_initialization = _early_release_initialization_code
else:
script_initialization = ''
zc.buildout.easy_install.sitepackage_safe_scripts( zc.buildout.easy_install.sitepackage_safe_scripts(
options['bin-directory'], ws, sys.executable, partsdir, options['bin-directory'], ws, sys.executable, partsdir,
reqs=['zc.buildout']) reqs=['zc.buildout'],
include_site_packages=_sys_executable_has_broken_dash_S,
script_initialization=script_initialization)
# Restart # Restart
args = map(zc.buildout.easy_install._safe_arg, sys.argv) args = map(zc.buildout.easy_install._safe_arg, sys.argv)
...@@ -951,7 +956,8 @@ class Buildout(UserDict.DictMixin): ...@@ -951,7 +956,8 @@ class Buildout(UserDict.DictMixin):
links = self['buildout'].get('find-links', '').split(), links = self['buildout'].get('find-links', '').split(),
index = self['buildout'].get('index'), index = self['buildout'].get('index'),
newest=self.newest, allow_hosts=self._allow_hosts, newest=self.newest, allow_hosts=self._allow_hosts,
include_site_packages=False) include_site_packages=_sys_executable_has_broken_dash_S,
prefer_final=not self.accept_buildout_test_releases)
# Clear cache because extensions might now let us read pages we # Clear cache because extensions might now let us read pages we
# couldn't read before. # couldn't read before.
...@@ -982,6 +988,7 @@ class Buildout(UserDict.DictMixin): ...@@ -982,6 +988,7 @@ class Buildout(UserDict.DictMixin):
setup = os.path.abspath(setup) setup = os.path.abspath(setup)
fd, tsetup = tempfile.mkstemp() fd, tsetup = tempfile.mkstemp()
exe = zc.buildout.easy_install._safe_arg(sys.executable)
try: try:
os.write(fd, zc.buildout.easy_install.runsetup_template % dict( os.write(fd, zc.buildout.easy_install.runsetup_template % dict(
setuptools=pkg_resources_loc, setuptools=pkg_resources_loc,
...@@ -995,14 +1002,10 @@ class Buildout(UserDict.DictMixin): ...@@ -995,14 +1002,10 @@ class Buildout(UserDict.DictMixin):
for a in args: for a in args:
arg_list.append(zc.buildout.easy_install._safe_arg(a)) arg_list.append(zc.buildout.easy_install._safe_arg(a))
subprocess.Popen( subprocess.Popen([exe] + list(tsetup) + arg_list).wait()
[zc.buildout.easy_install._safe_arg(sys.executable)]
+ list(tsetup)
+ arg_list
).wait()
else: else:
os.spawnl(os.P_WAIT, sys.executable, zc.buildout.easy_install._safe_arg (sys.executable), tsetup, os.spawnl(os.P_WAIT, sys.executable, exe, tsetup,
*[zc.buildout.easy_install._safe_arg(a) *[zc.buildout.easy_install._safe_arg(a)
for a in args]) for a in args])
finally: finally:
...@@ -1069,8 +1072,8 @@ def _install_and_load(spec, group, entry, buildout): ...@@ -1069,8 +1072,8 @@ def _install_and_load(spec, group, entry, buildout):
working_set=pkg_resources.working_set, working_set=pkg_resources.working_set,
newest=buildout.newest, newest=buildout.newest,
allow_hosts=buildout._allow_hosts, allow_hosts=buildout._allow_hosts,
include_site_packages=False, include_site_packages=_sys_executable_has_broken_dash_S,
) prefer_final=not buildout.accept_buildout_test_releases)
__doing__ = 'Loading %s recipe entry %s:%s.', group, spec, entry __doing__ = 'Loading %s recipe entry %s:%s.', group, spec, entry
return pkg_resources.load_entry_point( return pkg_resources.load_entry_point(
...@@ -1083,6 +1086,7 @@ def _install_and_load(spec, group, entry, buildout): ...@@ -1083,6 +1086,7 @@ def _install_and_load(spec, group, entry, buildout):
group, entry, spec, v) group, entry, spec, v)
raise raise
class Options(UserDict.DictMixin): class Options(UserDict.DictMixin):
def __init__(self, buildout, section, data): def __init__(self, buildout, section, data):
...@@ -1292,6 +1296,32 @@ class Options(UserDict.DictMixin): ...@@ -1292,6 +1296,32 @@ class Options(UserDict.DictMixin):
self.name) self.name)
return self._created return self._created
def query_bool(self, name, default=None):
"""Given a name, return a boolean value for that name.
``default``, if given, should be 'true', 'false', or None.
"""
if default is not None:
value = self.setdefault(name, default=default)
else:
value = self.get(name)
if value is None:
return value
return _convert_bool(name, value)
def get_bool(self, name):
"""Given a name, return a boolean value for that name.
"""
return _convert_bool(name, self[name])
def _convert_bool(name, value):
if value not in ('true', 'false'):
raise zc.buildout.UserError(
'Invalid value for %s option: %s' % (name, value))
else:
return value == 'true'
_spacey_nl = re.compile('[ \t\r\f\v]*\n[ \t\r\f\v\n]*' _spacey_nl = re.compile('[ \t\r\f\v]*\n[ \t\r\f\v\n]*'
'|' '|'
'^[ \t\r\f\v]+' '^[ \t\r\f\v]+'
...@@ -1514,6 +1544,13 @@ def _check_for_unused_options_in_section(buildout, section): ...@@ -1514,6 +1544,13 @@ def _check_for_unused_options_in_section(buildout, section):
% (section, ' '.join(map(repr, unused))) % (section, ' '.join(map(repr, unused)))
) )
_early_release_initialization_code = """\
sys.argv.insert(1, 'buildout:accept-buildout-test-releases=true')
print ('NOTE: Accepting early releases of build system packages. Rerun '
'bootstrap without --accept-buildout-test-releases (-t) to return to '
'default behavior.')
"""
_usage = """\ _usage = """\
Usage: buildout [options] [assignments] [command [command arguments]] Usage: buildout [options] [assignments] [command [command arguments]]
...@@ -1577,6 +1614,11 @@ Options: ...@@ -1577,6 +1614,11 @@ Options:
will be started. This is especially useful for debuging recipe will be started. This is especially useful for debuging recipe
problems. problems.
-s
Squelch warnings about using an executable with a broken -S
implementation.
Assignments are of the form: section:option=value and are used to Assignments are of the form: section:option=value and are used to
provide configuration options that override those given in the provide configuration options that override those given in the
configuration file. For example, to run the buildout in offline mode, configuration file. For example, to run the buildout in offline mode,
...@@ -1642,11 +1684,12 @@ def main(args=None): ...@@ -1642,11 +1684,12 @@ def main(args=None):
windows_restart = False windows_restart = False
user_defaults = True user_defaults = True
debug = False debug = False
ignore_broken_dash_s = False
while args: while args:
if args[0][0] == '-': if args[0][0] == '-':
op = orig_op = args.pop(0) op = orig_op = args.pop(0)
op = op[1:] op = op[1:]
while op and op[0] in 'vqhWUoOnNDA': while op and op[0] in 'vqhWUoOnNDAs':
if op[0] == 'v': if op[0] == 'v':
verbosity += 10 verbosity += 10
elif op[0] == 'q': elif op[0] == 'q':
...@@ -1665,6 +1708,8 @@ def main(args=None): ...@@ -1665,6 +1708,8 @@ def main(args=None):
options.append(('buildout', 'newest', 'false')) options.append(('buildout', 'newest', 'false'))
elif op[0] == 'D': elif op[0] == 'D':
debug = True debug = True
elif op[0] == 's':
ignore_broken_dash_s = True
else: else:
_help() _help()
op = op[1:] op = op[1:]
...@@ -1708,6 +1753,17 @@ def main(args=None): ...@@ -1708,6 +1753,17 @@ def main(args=None):
# The rest should be commands, so we'll stop here # The rest should be commands, so we'll stop here
break break
if verbosity < 0 or ignore_broken_dash_s:
broken_dash_S_filter_action = 'ignore'
elif verbosity == 0: # This is the default.
broken_dash_S_filter_action = 'once'
else:
broken_dash_S_filter_action = 'default'
warnings.filterwarnings(
broken_dash_S_filter_action,
re.escape(
zc.buildout.easy_install.BROKEN_DASH_S_WARNING),
UserWarning)
if verbosity: if verbosity:
options.append(('buildout', 'verbosity', str(verbosity))) options.append(('buildout', 'verbosity', str(verbosity)))
......
...@@ -729,6 +729,8 @@ COMMAND_LINE_VALUE). ...@@ -729,6 +729,8 @@ COMMAND_LINE_VALUE).
================== ==================
<BLANKLINE> <BLANKLINE>
[buildout] [buildout]
accept-buildout-test-releases= false
DEFAULT_VALUE
allow-hosts= * allow-hosts= *
DEFAULT_VALUE DEFAULT_VALUE
allow-picked-versions= true allow-picked-versions= true
...@@ -2227,6 +2229,7 @@ database is shown. ...@@ -2227,6 +2229,7 @@ database is shown.
<BLANKLINE> <BLANKLINE>
Configuration data: Configuration data:
[buildout] [buildout]
accept-buildout-test-releases = false
allow-hosts = * allow-hosts = *
allow-picked-versions = true allow-picked-versions = true
bin-directory = /sample-buildout/bin bin-directory = /sample-buildout/bin
...@@ -2484,25 +2487,33 @@ the buildout -o option. ...@@ -2484,25 +2487,33 @@ the buildout -o option.
Preferring Final Releases Preferring Final Releases
------------------------- -------------------------
Currently, when searching for new releases, the newest available Currently, when searching for new releases of your project's
release is used. This isn't usually ideal, as you may get a dependencies, the newest available release is used. This isn't usually
development release or alpha releases not ready to be widely used. ideal, as you may get a development release or alpha releases not ready
You can request that final releases be preferred using the prefer to be widely used. You can request that final releases be preferred
final option in the buildout section:: using the ``prefer-final`` option in the buildout section::
[buildout] [buildout]
... ...
prefer-final = true prefer-final = true
When the prefer-final option is set to true, then when searching for When the ``prefer-final`` option is set to true, then when searching for
new releases, final releases are preferred. If there are final new releases, final releases are preferred. If there are final
releases that satisfy distribution requirements, then those releases releases that satisfy distribution requirements, then those releases
are used even if newer non-final releases are available. The buildout are used even if newer non-final releases are available.
prefer-final option can be used to override this behavior.
In buildout version 2, all final releases will be preferred by
In buildout version 2, final releases will be preferred by default. default--that is ``prefer-final`` will also default to 'true'. You will
You will then need to use a false value for prefer-final to get the then need to use a 'false' value for ``prefer-final`` to get the newest
newest releases. releases.
A separate option controls the behavior of the build system itself.
When buildout looks for recipes, extensions, and for updates to itself,
it does prefer final releases by default, as of the 1.5.0 release. The
``accept-buildout-test-releases`` option will let you override this behavior.
However, it is typically changed by the --accept-buildout-test-releases
option to the bootstrap script, since bootstrapping is the first step to
selecting a buildout.
Finding distributions Finding distributions
--------------------- ---------------------
......
...@@ -5,12 +5,12 @@ Buildouts can be pretty complex. When things go wrong, it isn't ...@@ -5,12 +5,12 @@ Buildouts can be pretty complex. When things go wrong, it isn't
always obvious why. Errors can occur due to problems in user input or always obvious why. Errors can occur due to problems in user input or
due to bugs in zc.buildout or recipes. When an error occurs, Python's due to bugs in zc.buildout or recipes. When an error occurs, Python's
post-mortem debugger can be used to inspect the state of the buildout post-mortem debugger can be used to inspect the state of the buildout
or recipe code were there error occurred. To enable this, use the -D or recipe code where the error occurred. To enable this, use the -D
option to the buildout. Let's create a recipe that has a bug: option to the buildout. Let's create a recipe that has a bug:
>>> mkdir(sample_buildout, 'recipes') >>> mkdir(sample_buildout, 'recipes')
>>> write(sample_buildout, 'recipes', 'mkdir.py', >>> write(sample_buildout, 'recipes', 'mkdir.py',
... """ ... """
... import os, zc.buildout ... import os, zc.buildout
... ...
...@@ -35,7 +35,7 @@ option to the buildout. Let's create a recipe that has a bug: ...@@ -35,7 +35,7 @@ option to the buildout. Let's create a recipe that has a bug:
>>> write(sample_buildout, 'recipes', 'setup.py', >>> write(sample_buildout, 'recipes', 'setup.py',
... """ ... """
... from setuptools import setup ... from setuptools import setup
... ...
... setup(name = "recipes", ... setup(name = "recipes",
... entry_points = {'zc.buildout': ['mkdir = mkdir:Mkdir']}, ... entry_points = {'zc.buildout': ['mkdir = mkdir:Mkdir']},
... ) ... )
......
...@@ -20,7 +20,7 @@ testing repository. ...@@ -20,7 +20,7 @@ testing repository.
Turn on logging on this server so that we can see when eggs are pulled Turn on logging on this server so that we can see when eggs are pulled
from it. from it.
>>> get(link_server2 + 'enable_server_logging') >>> get(link_server2 + 'enable_server_logging')
GET 200 /enable_server_logging GET 200 /enable_server_logging
'' ''
...@@ -83,10 +83,9 @@ buildout to see where the egg comes from this time. ...@@ -83,10 +83,9 @@ buildout to see where the egg comes from this time.
... for egg in glob(join(sample_buildout, 'eggs', 'demoneeded*.egg')): ... for egg in glob(join(sample_buildout, 'eggs', 'demoneeded*.egg')):
... remove(sample_buildout, 'eggs', egg) ... remove(sample_buildout, 'eggs', egg)
>>> remove_demoneeded_egg() >>> remove_demoneeded_egg()
>>> print system(buildout) >>> print system(buildout) # doctest: +ELLIPSIS
Develop: '/sample-buildout/depdemo' Develop: '/sample-buildout/depdemo'
Updating eggs. ...
Couldn't find index page for 'demoneeded' (maybe misspelled?)
Getting distribution for 'demoneeded'. Getting distribution for 'demoneeded'.
While: While:
Updating eggs. Updating eggs.
......
...@@ -33,6 +33,7 @@ import shutil ...@@ -33,6 +33,7 @@ import shutil
import subprocess import subprocess
import sys import sys
import tempfile import tempfile
import warnings
import zc.buildout import zc.buildout
import zipimport import zipimport
...@@ -54,11 +55,22 @@ is_jython = sys.platform.startswith('java') ...@@ -54,11 +55,22 @@ is_jython = sys.platform.startswith('java')
is_distribute = ( is_distribute = (
pkg_resources.Requirement.parse('setuptools').key=='distribute') pkg_resources.Requirement.parse('setuptools').key=='distribute')
BROKEN_DASH_S_WARNING = (
'Buildout has been asked to exclude or limit site-packages so that '
'builds can be repeatable when using a system Python. However, '
'the chosen Python executable has a broken implementation of -S (see '
'https://bugs.launchpad.net/virtualenv/+bug/572545 for an example '
"problem) and this breaks buildout's ability to isolate site-packages. "
"If the executable already has a clean site-packages (e.g., "
"using virtualenv's ``--no-site-packages`` option) you may be getting "
'equivalent repeatability. To silence this warning, use the -s argument '
'to the buildout script. Alternatively, use a Python executable with a '
'working -S (such as a standard Python binary).')
if is_jython: if is_jython:
import java.lang.System import java.lang.System
jython_os_name = (java.lang.System.getProperties()['os.name']).lower() jython_os_name = (java.lang.System.getProperties()['os.name']).lower()
setuptools_loc = pkg_resources.working_set.find( setuptools_loc = pkg_resources.working_set.find(
pkg_resources.Requirement.parse('setuptools') pkg_resources.Requirement.parse('setuptools')
).location ).location
...@@ -71,6 +83,25 @@ buildout_and_setuptools_path = [setuptools_loc] ...@@ -71,6 +83,25 @@ buildout_and_setuptools_path = [setuptools_loc]
if os.path.normpath(setuptools_loc) != os.path.normpath(buildout_loc): if os.path.normpath(setuptools_loc) != os.path.normpath(buildout_loc):
buildout_and_setuptools_path.append(buildout_loc) buildout_and_setuptools_path.append(buildout_loc)
def _has_broken_dash_S(executable):
"""Detect https://bugs.launchpad.net/virtualenv/+bug/572545 ."""
# The first attempt here was to simply have the executable attempt to import
# ConfigParser and return the return code. That worked except for tests on
# Windows, where the return code was wrong for the fake Python executable
# generated by the virtualenv.txt test, apparently because setuptools' .exe
# file does not pass the -script.py's returncode back properly, at least in
# some circumstances. Therefore...print statements.
stdout, stderr = subprocess.Popen(
[executable, '-Sc',
'try:\n'
' import ConfigParser\n'
'except ImportError:\n'
' print 1\n'
'else:\n'
' print 0\n'],
stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()
return bool(int(stdout.strip()))
def _get_system_paths(executable): def _get_system_paths(executable):
"""Return lists of standard lib and site paths for executable. """Return lists of standard lib and site paths for executable.
""" """
...@@ -124,7 +155,8 @@ def _get_system_paths(executable): ...@@ -124,7 +155,8 @@ def _get_system_paths(executable):
return (stdlib, site_paths) return (stdlib, site_paths)
def _get_version_info(executable): def _get_version_info(executable):
cmd = [executable, '-Sc', 'import sys; print repr(sys.version_info)'] cmd = [executable, '-Sc',
'import sys; print(repr(tuple(x for x in sys.version_info)))']
_proc = subprocess.Popen( _proc = subprocess.Popen(
cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE) cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = _proc.communicate(); stdout, stderr = _proc.communicate();
...@@ -210,10 +242,10 @@ else: ...@@ -210,10 +242,10 @@ else:
_safe_arg = str _safe_arg = str
# The following string is used to run easy_install in # The following string is used to run easy_install in
# Installer._call_easy_install. It is started with python -S (that is, # Installer._call_easy_install. It is usually started with python -S
# don't import site at start). That flag, and all of the code in this # (that is, don't import site at start). That flag, and all of the code
# snippet above the last two lines, exist to work around a relatively rare # in this snippet above the last two lines, exist to work around a
# problem. If # relatively rare problem. If
# #
# - your buildout configuration is trying to install a package that is within # - your buildout configuration is trying to install a package that is within
# a namespace package, and # a namespace package, and
...@@ -239,7 +271,7 @@ else: ...@@ -239,7 +271,7 @@ else:
# for another description of the problem). Simply starting Python with # for another description of the problem). Simply starting Python with
# -S addresses the problem in Python 2.4 and 2.5, but Python 2.6's # -S addresses the problem in Python 2.4 and 2.5, but Python 2.6's
# distutils imports a value from the site module, so we unfortunately # distutils imports a value from the site module, so we unfortunately
# have to do more drastic surgery in the _easy_install_cmd code below. # have to do more drastic surgery in the _easy_install_preface code below.
# #
# Here's an example of the .pth files created by setuptools when using that # Here's an example of the .pth files created by setuptools when using that
# flag: # flag:
...@@ -265,17 +297,16 @@ else: ...@@ -265,17 +297,16 @@ else:
# unnecessary for site.py to preprocess these packages, so it should be # unnecessary for site.py to preprocess these packages, so it should be
# fine, as far as can be guessed as of this writing.) Finally, it # fine, as far as can be guessed as of this writing.) Finally, it
# imports easy_install and runs it. # imports easy_install and runs it.
_easy_install_preface = '''\
_easy_install_cmd = _safe_arg('''\
import sys,os;\ import sys,os;\
p = sys.path[:];\ p = sys.path[:];\
import site;\ import site;\
sys.path[:] = p;\ sys.path[:] = p;\
[sys.modules.pop(k) for k, v in sys.modules.items()\ [sys.modules.pop(k) for k, v in sys.modules.items()\
if hasattr(v, '__path__') and len(v.__path__)==1 and\ if hasattr(v, '__path__') and len(v.__path__)==1 and\
not os.path.exists(os.path.join(v.__path__[0],'__init__.py'))];\ not os.path.exists(os.path.join(v.__path__[0],'__init__.py'))];'''
from setuptools.command.easy_install import main;\ _easy_install_cmd = (
main()''') 'from setuptools.command.easy_install import main;main()')
class Installer: class Installer:
...@@ -302,7 +333,8 @@ class Installer: ...@@ -302,7 +333,8 @@ class Installer:
use_dependency_links=None, use_dependency_links=None,
allow_hosts=('*',), allow_hosts=('*',),
include_site_packages=None, include_site_packages=None,
allowed_eggs_from_site_packages=None allowed_eggs_from_site_packages=None,
prefer_final=None,
): ):
self._dest = dest self._dest = dest
self._allow_hosts = allow_hosts self._allow_hosts = allow_hosts
...@@ -316,12 +348,15 @@ class Installer: ...@@ -316,12 +348,15 @@ class Installer:
if use_dependency_links is not None: if use_dependency_links is not None:
self._use_dependency_links = use_dependency_links self._use_dependency_links = use_dependency_links
if prefer_final is not None:
self._prefer_final = prefer_final
self._links = links = list(_fix_file_links(links)) self._links = links = list(_fix_file_links(links))
if self._download_cache and (self._download_cache not in links): if self._download_cache and (self._download_cache not in links):
links.insert(0, self._download_cache) links.insert(0, self._download_cache)
self._index_url = index self._index_url = index
self._executable = executable self._executable = executable
self._has_broken_dash_S = _has_broken_dash_S(self._executable)
if always_unzip is not None: if always_unzip is not None:
self._always_unzip = always_unzip self._always_unzip = always_unzip
path = (path and path[:] or []) path = (path and path[:] or [])
...@@ -330,6 +365,17 @@ class Installer: ...@@ -330,6 +365,17 @@ class Installer:
if allowed_eggs_from_site_packages is not None: if allowed_eggs_from_site_packages is not None:
self._allowed_eggs_from_site_packages = tuple( self._allowed_eggs_from_site_packages = tuple(
allowed_eggs_from_site_packages) allowed_eggs_from_site_packages)
if self._has_broken_dash_S:
if (not self._include_site_packages or
self._allowed_eggs_from_site_packages != ('*',)):
# We can't do this if the executable has a broken -S.
warnings.warn(BROKEN_DASH_S_WARNING)
self._include_site_packages = True
self._allowed_eggs_from_site_packages = ('*',)
self._easy_install_cmd = _easy_install_cmd
else:
self._easy_install_cmd = _easy_install_preface + _easy_install_cmd
self._easy_install_cmd = _safe_arg(self._easy_install_cmd)
stdlib, self._site_packages = _get_system_paths(executable) stdlib, self._site_packages = _get_system_paths(executable)
version_info = _get_version_info(executable) version_info = _get_version_info(executable)
if version_info == sys.version_info: if version_info == sys.version_info:
...@@ -488,7 +534,9 @@ class Installer: ...@@ -488,7 +534,9 @@ class Installer:
try: try:
path = setuptools_loc path = setuptools_loc
args = ('-Sc', _easy_install_cmd, '-mUNxd', _safe_arg(tmp)) args = ('-c', self._easy_install_cmd, '-mUNxd', _safe_arg(tmp))
if not self._has_broken_dash_S:
args = ('-S',) + args
if self._always_unzip: if self._always_unzip:
args += ('-Z', ) args += ('-Z', )
level = logger.getEffectiveLevel() level = logger.getEffectiveLevel()
...@@ -785,7 +833,7 @@ class Installer: ...@@ -785,7 +833,7 @@ class Installer:
def _maybe_add_setuptools(self, ws, dist): def _maybe_add_setuptools(self, ws, dist):
if dist.has_metadata('namespace_packages.txt'): if dist.has_metadata('namespace_packages.txt'):
for r in dist.requires(): for r in dist.requires():
if r.project_name == 'setuptools': if r.project_name in ('setuptools', 'distribute'):
break break
else: else:
# We have a namespace package but no requirement for setuptools # We have a namespace package but no requirement for setuptools
...@@ -873,7 +921,13 @@ class Installer: ...@@ -873,7 +921,13 @@ class Installer:
dist = best[req.key] = env.best_match(req, ws) dist = best[req.key] = env.best_match(req, ws)
except pkg_resources.VersionConflict, err: except pkg_resources.VersionConflict, err:
raise VersionConflict(err, ws) raise VersionConflict(err, ws)
if dist is None: if dist is None or (
dist.location in self._site_packages and not
self.allow_site_package_egg(dist.project_name)):
# If we didn't find a distribution in the
# environment, or what we found is from site
# packages and not allowed to be there, try
# again.
if destination: if destination:
logger.debug('Getting required %r', str(req)) logger.debug('Getting required %r', str(req))
else: else:
...@@ -1020,13 +1074,14 @@ def install(specs, dest, ...@@ -1020,13 +1074,14 @@ def install(specs, dest,
executable=sys.executable, always_unzip=None, executable=sys.executable, always_unzip=None,
path=None, working_set=None, newest=True, versions=None, path=None, working_set=None, newest=True, versions=None,
use_dependency_links=None, allow_hosts=('*',), use_dependency_links=None, allow_hosts=('*',),
include_site_packages=None, allowed_eggs_from_site_packages=None): include_site_packages=None, allowed_eggs_from_site_packages=None,
installer = Installer(dest, links, index, executable, always_unzip, path, prefer_final=None):
newest, versions, use_dependency_links, installer = Installer(
allow_hosts=allow_hosts, dest, links, index, executable, always_unzip, path, newest,
include_site_packages=include_site_packages, versions, use_dependency_links, allow_hosts=allow_hosts,
allowed_eggs_from_site_packages= include_site_packages=include_site_packages,
allowed_eggs_from_site_packages) allowed_eggs_from_site_packages=allowed_eggs_from_site_packages,
prefer_final=prefer_final)
return installer.install(specs, working_set) return installer.install(specs, working_set)
...@@ -1035,11 +1090,11 @@ def build(spec, dest, build_ext, ...@@ -1035,11 +1090,11 @@ def build(spec, dest, build_ext,
executable=sys.executable, executable=sys.executable,
path=None, newest=True, versions=None, allow_hosts=('*',), path=None, newest=True, versions=None, allow_hosts=('*',),
include_site_packages=None, allowed_eggs_from_site_packages=None): include_site_packages=None, allowed_eggs_from_site_packages=None):
installer = Installer(dest, links, index, executable, True, path, newest, installer = Installer(
versions, allow_hosts=allow_hosts, dest, links, index, executable, True, path, newest, versions,
include_site_packages=include_site_packages, allow_hosts=allow_hosts,
allowed_eggs_from_site_packages= include_site_packages=include_site_packages,
allowed_eggs_from_site_packages) allowed_eggs_from_site_packages=allowed_eggs_from_site_packages)
return installer.build(spec, build_ext) return installer.build(spec, build_ext)
...@@ -1135,23 +1190,25 @@ def develop(setup, dest, ...@@ -1135,23 +1190,25 @@ def develop(setup, dest,
[f() for f in undo] [f() for f in undo]
def working_set(specs, executable, path, include_site_packages=None, def working_set(specs, executable, path, include_site_packages=None,
allowed_eggs_from_site_packages=None): allowed_eggs_from_site_packages=None, prefer_final=None):
return install( return install(
specs, None, executable=executable, path=path, specs, None, executable=executable, path=path,
include_site_packages=include_site_packages, include_site_packages=include_site_packages,
allowed_eggs_from_site_packages=allowed_eggs_from_site_packages) allowed_eggs_from_site_packages=allowed_eggs_from_site_packages,
prefer_final=prefer_final)
############################################################################ ############################################################################
# Script generation functions # Script generation functions
def scripts(reqs, working_set, executable, dest, def scripts(
scripts=None, reqs, working_set, executable, dest,
extra_paths=(), scripts=None,
arguments='', extra_paths=(),
interpreter=None, arguments='',
initialization='', interpreter=None,
relative_paths=False, initialization='',
): relative_paths=False,
):
"""Generate scripts and/or an interpreter. """Generate scripts and/or an interpreter.
See sitepackage_safe_scripts for a version that can be used with a Python See sitepackage_safe_scripts for a version that can be used with a Python
...@@ -1171,11 +1228,24 @@ def scripts(reqs, working_set, executable, dest, ...@@ -1171,11 +1228,24 @@ def scripts(reqs, working_set, executable, dest,
_pyscript(spath, sname, executable, rpsetup)) _pyscript(spath, sname, executable, rpsetup))
return generated return generated
# We need to give an alternate name to the ``scripts`` function so that it
# can be referenced within sitepackage_safe_scripts, which uses ``scripts``
# as an argument name.
_original_scripts_function = scripts
def sitepackage_safe_scripts( def sitepackage_safe_scripts(
dest, working_set, executable, site_py_dest, dest, working_set, executable, site_py_dest,
reqs=(), scripts=None, interpreter=None, extra_paths=(), reqs=(),
initialization='', include_site_packages=False, exec_sitecustomize=False, scripts=None,
relative_paths=False, script_arguments='', script_initialization=''): interpreter=None,
extra_paths=(),
initialization='',
include_site_packages=False,
exec_sitecustomize=False,
relative_paths=False,
script_arguments='',
script_initialization='',
):
"""Generate scripts and/or an interpreter from a system Python. """Generate scripts and/or an interpreter from a system Python.
This accomplishes the same job as the ``scripts`` function, above, This accomplishes the same job as the ``scripts`` function, above,
...@@ -1183,6 +1253,12 @@ def sitepackage_safe_scripts( ...@@ -1183,6 +1253,12 @@ def sitepackage_safe_scripts(
Python site packages, if desired, and choosing to execute the Python's Python site packages, if desired, and choosing to execute the Python's
sitecustomize. sitecustomize.
""" """
if _has_broken_dash_S(executable):
if not include_site_packages:
warnings.warn(BROKEN_DASH_S_WARNING)
return _original_scripts_function(
reqs, working_set, executable, dest, scripts, extra_paths,
script_arguments, interpreter, initialization, relative_paths)
generated = [] generated = []
generated.append(_generate_sitecustomize( generated.append(_generate_sitecustomize(
site_py_dest, executable, initialization, exec_sitecustomize)) site_py_dest, executable, initialization, exec_sitecustomize))
...@@ -1491,7 +1567,7 @@ def _get_module_file(executable, name): ...@@ -1491,7 +1567,7 @@ def _get_module_file(executable, name):
cmd = [executable, "-Sc", cmd = [executable, "-Sc",
"import imp; " "import imp; "
"fp, path, desc = imp.find_module(%r); " "fp, path, desc = imp.find_module(%r); "
"fp.close; " "fp.close(); "
"print path" % (name,)] "print path" % (name,)]
env = os.environ.copy() env = os.environ.copy()
# We need to make sure that PYTHONPATH, which will often be set to # We need to make sure that PYTHONPATH, which will often be set to
...@@ -1548,15 +1624,15 @@ def _generate_site(dest, working_set, executable, extra_paths=(), ...@@ -1548,15 +1624,15 @@ def _generate_site(dest, working_set, executable, extra_paths=(),
""" """
path = _get_path(working_set, extra_paths) path = _get_path(working_set, extra_paths)
site_path = os.path.join(dest, 'site.py') site_path = os.path.join(dest, 'site.py')
egg_path_string, preamble = _relative_path_and_setup( original_path_setup = preamble = ''
site_path, path, relative_paths, indent_level=2, omit_os_import=True)
if preamble:
preamble = '\n'.join(
[(line and ' %s' % (line,) or line)
for line in preamble.split('\n')])
original_path_setup = ''
if include_site_packages: if include_site_packages:
stdlib, site_paths = _get_system_paths(executable) stdlib, site_paths = _get_system_paths(executable)
# We want to make sure that paths from site-packages, such as those
# allowed by allowed_eggs_from_site_packages, always come last, or
# else site-packages paths may include packages that mask the eggs we
# really want.
path = [p for p in path if p not in site_paths]
# Now we set up the code we need.
original_path_setup = original_path_snippet % ( original_path_setup = original_path_snippet % (
_format_paths((repr(p) for p in site_paths), 2),) _format_paths((repr(p) for p in site_paths), 2),)
distribution = working_set.find( distribution = working_set.find(
...@@ -1570,10 +1646,19 @@ def _generate_site(dest, working_set, executable, extra_paths=(), ...@@ -1570,10 +1646,19 @@ def _generate_site(dest, working_set, executable, extra_paths=(),
relative_paths) relative_paths)
else: else:
location = repr(distribution.location) location = repr(distribution.location)
preamble += namespace_include_site_packages_setup % (location,) preamble = namespace_include_site_packages_setup % (location,)
original_path_setup = ( original_path_setup = (
addsitedir_namespace_originalpackages_snippet + addsitedir_namespace_originalpackages_snippet +
original_path_setup) original_path_setup)
else:
preamble = '\n setuptools_path = None'
egg_path_string, relative_preamble = _relative_path_and_setup(
site_path, path, relative_paths, indent_level=2, omit_os_import=True)
if relative_preamble:
relative_preamble = '\n'.join(
[(line and ' %s' % (line,) or line)
for line in relative_preamble.split('\n')])
preamble = relative_preamble + preamble
addsitepackages_marker = 'def addsitepackages(' addsitepackages_marker = 'def addsitepackages('
enableusersite_marker = 'ENABLE_USER_SITE = ' enableusersite_marker = 'ENABLE_USER_SITE = '
successful_rewrite = False successful_rewrite = False
...@@ -1616,7 +1701,8 @@ original_path_snippet = ''' ...@@ -1616,7 +1701,8 @@ original_path_snippet = '''
%s %s
] ]
for path in original_paths: for path in original_paths:
addsitedir(path, known_paths)''' if path == setuptools_path or path not in known_paths:
addsitedir(path, known_paths)'''
addsitepackages_script = '''\ addsitepackages_script = '''\
def addsitepackages(known_paths): def addsitepackages(known_paths):
......
...@@ -1277,6 +1277,7 @@ this is very straightforward. ...@@ -1277,6 +1277,7 @@ this is very straightforward.
"""Add site packages, as determined by zc.buildout. """Add site packages, as determined by zc.buildout.
<BLANKLINE> <BLANKLINE>
See original_addsitepackages, below, for the original version.""" See original_addsitepackages, below, for the original version."""
setuptools_path = None
buildout_paths = [ buildout_paths = [
'/interpreter/eggs/demo-0.3-pyN.N.egg', '/interpreter/eggs/demo-0.3-pyN.N.egg',
'/interpreter/eggs/demoneeded-1.1-pyN.N.egg' '/interpreter/eggs/demoneeded-1.1-pyN.N.egg'
...@@ -1291,7 +1292,8 @@ this is very straightforward. ...@@ -1291,7 +1292,8 @@ this is very straightforward.
... ...
] ]
for path in original_paths: for path in original_paths:
addsitedir(path, known_paths) if path == setuptools_path or path not in known_paths:
addsitedir(path, known_paths)
return known_paths return known_paths
<BLANKLINE> <BLANKLINE>
def original_addsitepackages(known_paths):... def original_addsitepackages(known_paths):...
...@@ -1368,7 +1370,8 @@ call to another text fixture to create. ...@@ -1368,7 +1370,8 @@ call to another text fixture to create.
... ...
] ]
for path in original_paths: for path in original_paths:
addsitedir(path, known_paths) if path == setuptools_path or path not in known_paths:
addsitedir(path, known_paths)
return known_paths return known_paths
<BLANKLINE> <BLANKLINE>
def original_addsitepackages(known_paths):... def original_addsitepackages(known_paths):...
...@@ -1432,7 +1435,8 @@ at that result. ...@@ -1432,7 +1435,8 @@ at that result.
... ...
] ]
for path in original_paths: for path in original_paths:
addsitedir(path, known_paths) if path == setuptools_path or path not in known_paths:
addsitedir(path, known_paths)
return known_paths return known_paths
<BLANKLINE> <BLANKLINE>
def original_addsitepackages(known_paths):... def original_addsitepackages(known_paths):...
......
...@@ -596,7 +596,7 @@ else: ...@@ -596,7 +596,7 @@ else:
sep = re.escape(os.path.sep) sep = re.escape(os.path.sep)
normalize_path = ( normalize_path = (
re.compile( re.compile(
r'''[^'" \t\n\r]+%(sep)s_[Tt][Ee][Ss][Tt]_%(sep)s([^"' \t\n\r]+)''' r'''[^'" \t\n\r!]+%(sep)s_[Tt][Ee][Ss][Tt]_%(sep)s([^"' \t\n\r]+)'''
% dict(sep=sep)), % dict(sep=sep)),
_normalize_path, _normalize_path,
) )
......
...@@ -15,7 +15,7 @@ The handers before calling set up are: ...@@ -15,7 +15,7 @@ The handers before calling set up are:
>>> len(logging.getLogger().handlers) >>> len(logging.getLogger().handlers)
1 1
>>> logging.getLogger().handlers # doctest: +ELLIPSIS >>> logging.getLogger().handlers # doctest: +ELLIPSIS
[<zope.testrunner.logsupport.NullHandler instance at ...>] [<zope...testrunner.logsupport.NullHandler instance at ...>]
After calling it, a ``logging.StreamHandler`` was added: After calling it, a ``logging.StreamHandler`` was added:
...@@ -27,7 +27,7 @@ After calling it, a ``logging.StreamHandler`` was added: ...@@ -27,7 +27,7 @@ After calling it, a ``logging.StreamHandler`` was added:
>>> len(logging.getLogger().handlers) >>> len(logging.getLogger().handlers)
2 2
>>> logging.getLogger().handlers # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE >>> logging.getLogger().handlers # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
[<zope.testrunner.logsupport.NullHandler instance at ...>, [<zope...testrunner.logsupport.NullHandler instance at ...>,
<logging.StreamHandler instance at ...>] <logging.StreamHandler instance at ...>]
But tear down removes the new logging handler: But tear down removes the new logging handler:
...@@ -36,4 +36,4 @@ But tear down removes the new logging handler: ...@@ -36,4 +36,4 @@ But tear down removes the new logging handler:
>>> len(logging.getLogger().handlers) >>> len(logging.getLogger().handlers)
1 1
>>> logging.getLogger().handlers # doctest: +ELLIPSIS >>> logging.getLogger().handlers # doctest: +ELLIPSIS
[<zope.testrunner.logsupport.NullHandler instance at ...>] [<zope...testrunner.logsupport.NullHandler instance at ...>]
...@@ -2255,6 +2255,75 @@ include-site-packages. ...@@ -2255,6 +2255,75 @@ include-site-packages.
""" """
def allowed_eggs_from_site_packages_dependencies_bugfix():
"""
If you specify that a package with a dependency may come from site-packages,
that doesn't mean that the dependency may come from site-packages. This
is a test for a bug fix to verify that this is true.
>>> py_path, site_packages_path = make_py()
>>> create_sample_sys_install(site_packages_path)
>>> interpreter_dir = tmpdir('interpreter')
>>> interpreter_parts_dir = os.path.join(
... interpreter_dir, 'parts', 'interpreter')
>>> interpreter_bin_dir = os.path.join(interpreter_dir, 'bin')
>>> mkdir(interpreter_bin_dir)
>>> mkdir(interpreter_dir, 'eggs')
>>> mkdir(interpreter_dir, 'parts')
>>> mkdir(interpreter_parts_dir)
>>> ws = zc.buildout.easy_install.install(
... ['demo'], join(interpreter_dir, 'eggs'), executable=py_path,
... links=[link_server], index=link_server+'index/',
... allowed_eggs_from_site_packages=['demo'])
>>> [dist.project_name for dist in ws]
['demo', 'demoneeded']
>>> from pprint import pprint
>>> pprint([dist.location for dist in ws])
['/executable_buildout/site-packages',
'/interpreter/eggs/demoneeded-1.1-pyN.N.egg']
"""
def allowed_eggs_from_site_packages_bug_592524():
"""
When we use allowed_eggs_from_site_packages, we need to make sure that the
site-packages paths are not inserted with the normal egg paths. They already
included at the end, and including them along with the normal egg paths will
possibly mask subsequent egg paths. This affects interpreters and scripts
generated by sitepackage_safe_scripts.
Our "py_path" has the "demoneeded" and "demo" packages available.
>>> py_path, site_packages_path = make_py()
>>> create_sample_sys_install(site_packages_path)
>>> interpreter_dir = tmpdir('interpreter')
>>> interpreter_parts_dir = os.path.join(
... interpreter_dir, 'parts', 'interpreter')
>>> interpreter_bin_dir = os.path.join(interpreter_dir, 'bin')
>>> mkdir(interpreter_bin_dir)
>>> mkdir(interpreter_dir, 'eggs')
>>> mkdir(interpreter_dir, 'parts')
>>> mkdir(interpreter_parts_dir)
>>> ws = zc.buildout.easy_install.install(
... ['demo', 'other'], join(interpreter_dir, 'eggs'), executable=py_path,
... links=[link_server], index=link_server+'index/',
... allowed_eggs_from_site_packages=['demo'])
>>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
... interpreter_bin_dir, ws, py_path, interpreter_parts_dir,
... interpreter='py', include_site_packages=True)
Now we will look at the paths in the site.py we generated. Notice that the
site-packages are at the end. They were not before this bugfix.
>>> test = 'import pprint, sys; pprint.pprint(sys.path[-4:])'
>>> print call_py(join(interpreter_bin_dir, 'py'), test)
['/interpreter/eggs/other-1.0-pyN.N.egg',
'/interpreter/eggs/demoneeded-1.1-pyN.N.egg',
'/executable_buildout/eggs/setuptools-0.0-pyN.N.egg',
'/executable_buildout/site-packages']
<BLANKLINE>
"""
def subprocesses_have_same_environment_by_default(): def subprocesses_have_same_environment_by_default():
""" """
The scripts generated by sitepackage_safe_scripts set the PYTHONPATH so that, The scripts generated by sitepackage_safe_scripts set the PYTHONPATH so that,
...@@ -3057,6 +3126,107 @@ We get an error if we specify anything but true or false: ...@@ -3057,6 +3126,107 @@ We get an error if we specify anything but true or false:
""" """
def buildout_prefer_final_build_system_option():
"""
The accept-buildout-test-releases buildout option can be used for overriding
the default preference for final distributions for recipes, buildout
extensions, and buildout itself. It is usually controlled via the bootstrap
script rather than in the configuration file, but we will test the machinery
using the file.
Set up. This creates sdists for demorecipe 1.0 and 1.1b1, and for
demoextension 1.0 and 1.1b1.
>>> create_sample_recipe_sdists(sample_eggs)
>>> create_sample_extension_sdists(sample_eggs)
The default is accept-buildout-test-releases = false:
>>> write('buildout.cfg',
... '''
... [buildout]
... parts = demo
... find-links = %(link_server)s
... extensions = demoextension
...
... [demo]
... recipe = demorecipe
... ''' % globals())
>>> print system(buildout+' -v'), # doctest: +ELLIPSIS
Installing ...
Picked: demoextension = 1.0
...
Picked: demorecipe = 1.0
...
Here we see that the final versions of demorecipe and demoextension were used.
We get the same behavior if we explicitly state that
accept-buildout-test-releases = false.
>>> write('buildout.cfg',
... '''
... [buildout]
... parts = demo
... find-links = %(link_server)s
... extensions = demoextension
... accept-buildout-test-releases = false
...
... [demo]
... recipe = demorecipe
... ''' % globals())
>>> print system(buildout+' -v'), # doctest: +ELLIPSIS
Installing ...
Picked: demoextension = 1.0
...
Picked: demorecipe = 1.0
...
If we specify accept-buildout-test-releases = true, we'll get the newest
distributions in the build system:
>>> write('buildout.cfg',
... '''
... [buildout]
... parts = demo
... find-links = %(link_server)s
... extensions = demoextension
... accept-buildout-test-releases = true
...
... [demo]
... recipe = demorecipe
... ''' % globals())
>>> print system(buildout+' -v'), # doctest: +ELLIPSIS
Installing ...
Picked: demoextension = 1.1b1
...
Picked: demorecipe = 1.1b1
...
We get an error if we specify anything but true or false:
>>> write('buildout.cfg',
... '''
... [buildout]
... parts = demo
... find-links = %(link_server)s
... extensions = demoextension
... accept-buildout-test-releases = no
...
... [demo]
... recipe = demorecipe
... ''' % globals())
>>> print system(buildout+' -v'), # doctest: +ELLIPSIS
While:
Initializing.
Error: Invalid value for accept-buildout-test-releases option: no
"""
def develop_with_modules(): def develop_with_modules():
""" """
Distribution setup scripts can import modules in the distribution directory: Distribution setup scripts can import modules in the distribution directory:
...@@ -3423,6 +3593,68 @@ def create_sample_namespace_eggs(dest, site_packages_path=None): ...@@ -3423,6 +3593,68 @@ def create_sample_namespace_eggs(dest, site_packages_path=None):
finally: finally:
shutil.rmtree(tmp) shutil.rmtree(tmp)
def create_sample_extension_sdists(dest):
from zc.buildout.testing import write, mkdir
name = 'demoextension'
for version in ('1.0', '1.1b1'):
tmp = tempfile.mkdtemp()
try:
write(tmp, 'README.txt', '')
write(tmp, name + '.py',
"def ext(buildout):\n"
" pass\n"
"def unload(buildout):\n"
" pass\n"
% locals())
write(tmp, 'setup.py',
"from setuptools import setup\n"
"setup(\n"
" name = %(name)r,\n"
" py_modules = [%(name)r],\n"
" entry_points = {\n"
" 'zc.buildout.extension': "
"['ext = %(name)s:ext'],\n"
" 'zc.buildout.unloadextension': "
"['ext = %(name)s:unload'],\n"
" },\n"
" zip_safe=True, version=%(version)r,\n"
" author='bob', url='bob', author_email='bob')\n"
% locals())
zc.buildout.testing.sdist(tmp, dest)
finally:
shutil.rmtree(tmp)
def create_sample_recipe_sdists(dest):
from zc.buildout.testing import write, mkdir
name = 'demorecipe'
for version in ('1.0', '1.1b1'):
tmp = tempfile.mkdtemp()
try:
write(tmp, 'README.txt', '')
write(tmp, name + '.py',
"import logging, os, zc.buildout\n"
"class Demorecipe:\n"
" def __init__(self, buildout, name, options):\n"
" self.name, self.options = name, options\n"
" def install(self):\n"
" return ()\n"
" def update(self):\n"
" pass\n"
% locals())
write(tmp, 'setup.py',
"from setuptools import setup\n"
"setup(\n"
" name = %(name)r,\n"
" py_modules = [%(name)r],\n"
" entry_points = {'zc.buildout': "
"['default = %(name)s:Demorecipe']},\n"
" zip_safe=True, version=%(version)r,\n"
" author='bob', url='bob', author_email='bob')\n"
% locals())
zc.buildout.testing.sdist(tmp, dest)
finally:
shutil.rmtree(tmp)
def _write_eggrecipedemoneeded(tmp, minor_version, suffix=''): def _write_eggrecipedemoneeded(tmp, minor_version, suffix=''):
from zc.buildout.testing import write from zc.buildout.testing import write
write(tmp, 'README.txt', '') write(tmp, 'README.txt', '')
...@@ -3570,37 +3802,33 @@ def easy_install_SetUp(test): ...@@ -3570,37 +3802,33 @@ def easy_install_SetUp(test):
egg_parse = re.compile('([0-9a-zA-Z_.]+)-([0-9a-zA-Z_.]+)-py(\d[.]\d).egg$' egg_parse = re.compile('([0-9a-zA-Z_.]+)-([0-9a-zA-Z_.]+)-py(\d[.]\d).egg$'
).match ).match
def makeNewRelease(project, ws, dest): def makeNewRelease(project, ws, dest, version='99.99'):
dist = ws.find(pkg_resources.Requirement.parse(project)) dist = ws.find(pkg_resources.Requirement.parse(project))
eggname, oldver, pyver = egg_parse( eggname, oldver, pyver = egg_parse(
os.path.basename(dist.location) os.path.basename(dist.location)
).groups() ).groups()
dest = os.path.join(dest, "%s-99.99-py%s.egg" % (eggname, pyver)) dest = os.path.join(dest, "%s-%s-py%s.egg" % (eggname, version, pyver))
if os.path.isfile(dist.location): if os.path.isfile(dist.location):
shutil.copy(dist.location, dest) shutil.copy(dist.location, dest)
zip = zipfile.ZipFile(dest, 'a') zip = zipfile.ZipFile(dest, 'a')
zip.writestr( zip.writestr(
'EGG-INFO/PKG-INFO', 'EGG-INFO/PKG-INFO',
zip.read('EGG-INFO/PKG-INFO').replace("Version: %s" % oldver, zip.read('EGG-INFO/PKG-INFO').replace("Version: %s" % oldver,
"Version: 99.99") "Version: %s" % version)
) )
zip.close() zip.close()
else: else:
shutil.copytree(dist.location, dest) shutil.copytree(dist.location, dest)
info_path = os.path.join(dest, 'EGG-INFO', 'PKG-INFO') info_path = os.path.join(dest, 'EGG-INFO', 'PKG-INFO')
info = open(info_path).read().replace("Version: %s" % oldver, info = open(info_path).read().replace("Version: %s" % oldver,
"Version: 99.99") "Version: %s" % version)
open(info_path, 'w').write(info) open(info_path, 'w').write(info)
def getWorkingSetWithBuildoutEgg(test):
def updateSetup(test):
zc.buildout.testing.buildoutSetUp(test)
new_releases = test.globs['tmpdir']('new_releases')
test.globs['new_releases'] = new_releases
sample_buildout = test.globs['sample_buildout'] sample_buildout = test.globs['sample_buildout']
eggs = os.path.join(sample_buildout, 'eggs') eggs = os.path.join(sample_buildout, 'eggs')
# If the zc.buildout dist is a develo dist, convert it to a # If the zc.buildout dist is a develop dist, convert it to a
# regular egg in the sample buildout # regular egg in the sample buildout
req = pkg_resources.Requirement.parse('zc.buildout') req = pkg_resources.Requirement.parse('zc.buildout')
dist = pkg_resources.working_set.find(req) dist = pkg_resources.working_set.find(req)
...@@ -3630,9 +3858,16 @@ def updateSetup(test): ...@@ -3630,9 +3858,16 @@ def updateSetup(test):
os.path.join(sample_buildout, 'bin')) os.path.join(sample_buildout, 'bin'))
else: else:
ws = pkg_resources.working_set ws = pkg_resources.working_set
return ws
def updateSetup(test):
zc.buildout.testing.buildoutSetUp(test)
new_releases = test.globs['tmpdir']('new_releases')
test.globs['new_releases'] = new_releases
ws = getWorkingSetWithBuildoutEgg(test)
# now let's make the new releases # now let's make the new releases
makeNewRelease('zc.buildout', ws, new_releases) makeNewRelease('zc.buildout', ws, new_releases)
makeNewRelease('zc.buildout', ws, new_releases, '100.0b1')
os.mkdir(os.path.join(new_releases, 'zc.buildout')) os.mkdir(os.path.join(new_releases, 'zc.buildout'))
if zc.buildout.easy_install.is_distribute: if zc.buildout.easy_install.is_distribute:
makeNewRelease('distribute', ws, new_releases) makeNewRelease('distribute', ws, new_releases)
...@@ -3641,6 +3876,13 @@ def updateSetup(test): ...@@ -3641,6 +3876,13 @@ def updateSetup(test):
makeNewRelease('setuptools', ws, new_releases) makeNewRelease('setuptools', ws, new_releases)
os.mkdir(os.path.join(new_releases, 'setuptools')) os.mkdir(os.path.join(new_releases, 'setuptools'))
def bootstrapSetup(test):
easy_install_SetUp(test)
sample_eggs = test.globs['sample_eggs']
ws = getWorkingSetWithBuildoutEgg(test)
makeNewRelease('zc.buildout', ws, sample_eggs)
makeNewRelease('zc.buildout', ws, sample_eggs, '100.0b1')
os.environ['bootstrap-testing-find-links'] = test.globs['link_server']
normalize_bang = ( normalize_bang = (
re.compile(re.escape('#!'+ re.compile(re.escape('#!'+
...@@ -3648,6 +3890,19 @@ normalize_bang = ( ...@@ -3648,6 +3890,19 @@ normalize_bang = (
'#!/usr/local/bin/python2.4', '#!/usr/local/bin/python2.4',
) )
hide_distribute_additions = (re.compile('install_dir .+\n'), '')
hide_zip_safe_message = (
# This comes in a different place in the output in Python 2.7. It's not
# important to our tests. Hide it.
re.compile(
'((?<=\n)\n)?zip_safe flag not set; analyzing archive contents...\n'),
'')
hide_first_index_page_message = (
# This comes in a different place in the output in Python 2.7. It's not
# important to our tests. Hide it.
re.compile(
"Couldn't find index page for '[^']+' \(maybe misspelled\?\)\n"),
'')
def test_suite(): def test_suite():
test_suite = [ test_suite = [
doctest.DocFileSuite( doctest.DocFileSuite(
...@@ -3655,44 +3910,47 @@ def test_suite(): ...@@ -3655,44 +3910,47 @@ def test_suite():
setUp=zc.buildout.testing.buildoutSetUp, setUp=zc.buildout.testing.buildoutSetUp,
tearDown=zc.buildout.testing.buildoutTearDown, tearDown=zc.buildout.testing.buildoutTearDown,
checker=renormalizing.RENormalizing([ checker=renormalizing.RENormalizing([
zc.buildout.testing.normalize_path, zc.buildout.testing.normalize_path,
zc.buildout.testing.normalize_endings, zc.buildout.testing.normalize_endings,
zc.buildout.testing.normalize_script, zc.buildout.testing.normalize_script,
zc.buildout.testing.normalize_egg_py, zc.buildout.testing.normalize_egg_py,
(re.compile('__buildout_signature__ = recipes-\S+'), zc.buildout.tests.hide_distribute_additions,
'__buildout_signature__ = recipes-SSSSSSSSSSS'), hide_zip_safe_message,
(re.compile('executable = [\S ]+python\S*', re.I), (re.compile('__buildout_signature__ = recipes-\S+'),
'executable = python'), '__buildout_signature__ = recipes-SSSSSSSSSSS'),
(re.compile('[-d] (setuptools|distribute)-\S+[.]egg'), (re.compile('executable = [\S ]+python\S*', re.I),
'setuptools.egg'), 'executable = python'),
(re.compile('zc.buildout(-\S+)?[.]egg(-link)?'), (re.compile('[-d] (setuptools|distribute)-\S+[.]egg'),
'zc.buildout.egg'), 'setuptools.egg'),
(re.compile('creating \S*setup.cfg'), 'creating setup.cfg'), (re.compile('zc.buildout(-\S+)?[.]egg(-link)?'),
(re.compile('hello\%ssetup' % os.path.sep), 'hello/setup'), 'zc.buildout.egg'),
(re.compile('Picked: (\S+) = \S+'), (re.compile('creating \S*setup.cfg'), 'creating setup.cfg'),
'Picked: \\1 = V.V'), (re.compile('hello\%ssetup' % os.path.sep), 'hello/setup'),
(re.compile(r'We have a develop egg: zc.buildout (\S+)'), (re.compile('Picked: (\S+) = \S+'),
'We have a develop egg: zc.buildout X.X.'), 'Picked: \\1 = V.V'),
(re.compile(r'\\[\\]?'), '/'), (re.compile(r'We have a develop egg: zc.buildout (\S+)'),
(re.compile('WindowsError'), 'OSError'), 'We have a develop egg: zc.buildout X.X.'),
(re.compile(r'\[Error \d+\] Cannot create a file ' (re.compile(r'\\[\\]?'), '/'),
r'when that file already exists: '), (re.compile('WindowsError'), 'OSError'),
'[Errno 17] File exists: ' (re.compile(r'\[Error \d+\] Cannot create a file '
), r'when that file already exists: '),
(re.compile('distribute'), 'setuptools'), '[Errno 17] File exists: '
]) ),
(re.compile('distribute'), 'setuptools'),
])
), ),
doctest.DocFileSuite( doctest.DocFileSuite(
'debugging.txt', 'debugging.txt',
setUp=zc.buildout.testing.buildoutSetUp, setUp=zc.buildout.testing.buildoutSetUp,
tearDown=zc.buildout.testing.buildoutTearDown, tearDown=zc.buildout.testing.buildoutTearDown,
checker=renormalizing.RENormalizing([ checker=renormalizing.RENormalizing([
zc.buildout.testing.normalize_path, zc.buildout.testing.normalize_path,
zc.buildout.testing.normalize_endings, zc.buildout.testing.normalize_endings,
(re.compile(r'\S+buildout.py'), 'buildout.py'), zc.buildout.tests.hide_distribute_additions,
(re.compile(r'line \d+'), 'line NNN'), (re.compile(r'\S+buildout.py'), 'buildout.py'),
(re.compile(r'py\(\d+\)'), 'py(NNN)'), (re.compile(r'line \d+'), 'line NNN'),
]) (re.compile(r'py\(\d+\)'), 'py(NNN)'),
])
), ),
doctest.DocFileSuite( doctest.DocFileSuite(
...@@ -3700,28 +3958,29 @@ def test_suite(): ...@@ -3700,28 +3958,29 @@ def test_suite():
setUp=updateSetup, setUp=updateSetup,
tearDown=zc.buildout.testing.buildoutTearDown, tearDown=zc.buildout.testing.buildoutTearDown,
checker=renormalizing.RENormalizing([ checker=renormalizing.RENormalizing([
zc.buildout.testing.normalize_path, zc.buildout.testing.normalize_path,
zc.buildout.testing.normalize_endings, zc.buildout.testing.normalize_endings,
zc.buildout.testing.normalize_script, zc.buildout.testing.normalize_script,
zc.buildout.testing.normalize_egg_py, zc.buildout.testing.normalize_egg_py,
normalize_bang, normalize_bang,
(re.compile('99[.]99'), 'NINETYNINE.NINETYNINE'), zc.buildout.tests.hide_distribute_additions,
(re.compile('(zc.buildout|setuptools)-\d+[.]\d+\S*' (re.compile('99[.]99'), 'NINETYNINE.NINETYNINE'),
'-py\d.\d.egg'), (re.compile('(zc.buildout|setuptools)-\d+[.]\d+\S*'
'\\1.egg'), '-py\d.\d.egg'),
(re.compile('distribute-\d+[.]\d+\S*' '\\1.egg'),
'-py\d.\d.egg'), (re.compile('distribute-\d+[.]\d+\S*'
'setuptools.egg'), '-py\d.\d.egg'),
(re.compile('(zc.buildout|setuptools)( version)? \d+[.]\d+\S*'), 'setuptools.egg'),
'\\1 V.V'), (re.compile('(zc.buildout|setuptools)( version)? \d+[.]\d+\S*'),
(re.compile('distribute( version)? \d+[.]\d+\S*'), '\\1 V.V'),
'setuptools V.V'), (re.compile('distribute( version)? \d+[.]\d+\S*'),
(re.compile('[-d] (setuptools|distribute)'), '- setuptools'), 'setuptools V.V'),
(re.compile('distribute'), 'setuptools'), (re.compile('[-d] (setuptools|distribute)'), '- setuptools'),
(re.compile("\nUnused options for buildout: " (re.compile('distribute'), 'setuptools'),
"'(distribute|setuptools)\-version'\."), (re.compile("\nUnused options for buildout: "
'') "'(distribute|setuptools)\-version'\."),
]) '')
])
), ),
doctest.DocFileSuite( doctest.DocFileSuite(
...@@ -3730,28 +3989,30 @@ def test_suite(): ...@@ -3730,28 +3989,30 @@ def test_suite():
setUp=easy_install_SetUp, setUp=easy_install_SetUp,
tearDown=zc.buildout.testing.buildoutTearDown, tearDown=zc.buildout.testing.buildoutTearDown,
checker=renormalizing.RENormalizing([ checker=renormalizing.RENormalizing([
zc.buildout.testing.normalize_path, zc.buildout.testing.normalize_path,
zc.buildout.testing.normalize_endings, zc.buildout.testing.normalize_endings,
zc.buildout.testing.normalize_script, zc.buildout.testing.normalize_script,
zc.buildout.testing.normalize_egg_py, zc.buildout.testing.normalize_egg_py,
normalize_bang, normalize_bang,
(re.compile('extdemo[.]pyd'), 'extdemo.so'), hide_first_index_page_message,
(re.compile('[-d] (setuptools|distribute)-\S+[.]egg'), zc.buildout.tests.hide_distribute_additions,
'setuptools.egg'), (re.compile('extdemo[.]pyd'), 'extdemo.so'),
(re.compile(r'\\[\\]?'), '/'), (re.compile('[-d] (setuptools|distribute)-\S+[.]egg'),
(re.compile(r'\#!\S+\bpython\S*'), '#!/usr/bin/python'), 'setuptools.egg'),
# Normalize generate_script's Windows interpreter to UNIX: (re.compile(r'\\[\\]?'), '/'),
(re.compile(r'\nimport subprocess\n'), '\n'), (re.compile(r'\#!\S+\bpython\S*'), '#!/usr/bin/python'),
(re.compile('subprocess\\.call\\(argv, env=environ\\)'), # Normalize generate_script's Windows interpreter to UNIX:
'os.execve(sys.executable, argv, environ)'), (re.compile(r'\nimport subprocess\n'), '\n'),
(re.compile('distribute'), 'setuptools'), (re.compile('subprocess\\.call\\(argv, env=environ\\)'),
# Distribute unzips eggs by default. 'os.execve(sys.executable, argv, environ)'),
(re.compile('\- demoneeded'), 'd demoneeded'), (re.compile('distribute'), 'setuptools'),
]+(sys.version_info < (2, 5) and [ # Distribute unzips eggs by default.
(re.compile('.*No module named runpy.*', re.S), ''), (re.compile('\- demoneeded'), 'd demoneeded'),
(re.compile('.*usage: pdb.py scriptfile .*', re.S), ''), ]+(sys.version_info < (2, 5) and [
(re.compile('.*Error: what does not exist.*', re.S), ''), (re.compile('.*No module named runpy.*', re.S), ''),
] or [])), (re.compile('.*usage: pdb.py scriptfile .*', re.S), ''),
(re.compile('.*Error: what does not exist.*', re.S), ''),
] or [])),
), ),
doctest.DocFileSuite( doctest.DocFileSuite(
...@@ -3772,35 +4033,37 @@ def test_suite(): ...@@ -3772,35 +4033,37 @@ def test_suite():
setUp=easy_install_SetUp, setUp=easy_install_SetUp,
tearDown=zc.buildout.testing.buildoutTearDown, tearDown=zc.buildout.testing.buildoutTearDown,
checker=renormalizing.RENormalizing([ checker=renormalizing.RENormalizing([
zc.buildout.testing.normalize_path, zc.buildout.testing.normalize_path,
zc.buildout.testing.normalize_endings, zc.buildout.testing.normalize_endings,
zc.buildout.testing.normalize_script, zc.buildout.testing.normalize_script,
zc.buildout.testing.normalize_egg_py, zc.buildout.testing.normalize_egg_py,
(re.compile("buildout: Running \S*setup.py"), zc.buildout.tests.hide_distribute_additions,
'buildout: Running setup.py'), hide_first_index_page_message,
(re.compile('(setuptools|distribute)-\S+-'), (re.compile("buildout: Running \S*setup.py"),
'setuptools.egg'), 'buildout: Running setup.py'),
(re.compile('zc.buildout-\S+-'), (re.compile('(setuptools|distribute)-\S+-'),
'zc.buildout.egg'), 'setuptools.egg'),
(re.compile('File "\S+one.py"'), (re.compile('zc.buildout-\S+-'),
'File "one.py"'), 'zc.buildout.egg'),
(re.compile(r'We have a develop egg: (\S+) (\S+)'), (re.compile('File "\S+one.py"'),
r'We have a develop egg: \1 V'), 'File "one.py"'),
(re.compile('Picked: (setuptools|distribute) = \S+'), (re.compile(r'We have a develop egg: (\S+) (\S+)'),
'Picked: setuptools = V'), r'We have a develop egg: \1 V'),
(re.compile(r'\\[\\]?'), '/'), (re.compile('Picked: (setuptools|distribute) = \S+'),
(re.compile( 'Picked: setuptools = V'),
'-q develop -mxN -d "/sample-buildout/develop-eggs'), (re.compile(r'\\[\\]?'), '/'),
'-q develop -mxN -d /sample-buildout/develop-eggs' (re.compile(
), '-q develop -mxN -d "/sample-buildout/develop-eggs'),
(re.compile(r'^[*]...'), '...'), '-q develop -mxN -d /sample-buildout/develop-eggs'
# for bug_92891_bootstrap_crashes_with_egg_recipe_in_buildout_section ),
(re.compile(r"Unused options for buildout: 'eggs' 'scripts'\."), (re.compile(r'^[*]...'), '...'),
"Unused options for buildout: 'scripts' 'eggs'."), # for bug_92891_bootstrap_crashes_with_egg_recipe_in_buildout_section
(re.compile('distribute'), 'setuptools'), (re.compile(r"Unused options for buildout: 'eggs' 'scripts'\."),
# Distribute unzips eggs by default. "Unused options for buildout: 'scripts' 'eggs'."),
(re.compile('\- demoneeded'), 'd demoneeded'), (re.compile('distribute'), 'setuptools'),
]), # Distribute unzips eggs by default.
(re.compile('\- demoneeded'), 'd demoneeded'),
]),
), ),
zc.buildout.testselectingpython.test_suite(), zc.buildout.testselectingpython.test_suite(),
zc.buildout.rmtree.test_suite(), zc.buildout.rmtree.test_suite(),
...@@ -3809,31 +4072,42 @@ def test_suite(): ...@@ -3809,31 +4072,42 @@ def test_suite():
setUp=zc.buildout.testing.buildoutSetUp, setUp=zc.buildout.testing.buildoutSetUp,
tearDown=zc.buildout.testing.buildoutTearDown, tearDown=zc.buildout.testing.buildoutTearDown,
checker=renormalizing.RENormalizing([ checker=renormalizing.RENormalizing([
zc.buildout.testing.normalize_path, zc.buildout.testing.normalize_path,
zc.buildout.testing.normalize_endings, zc.buildout.testing.normalize_endings,
zc.buildout.testing.normalize_script, zc.buildout.testing.normalize_script,
zc.buildout.testing.normalize_egg_py, zc.buildout.testing.normalize_egg_py,
(re.compile('__buildout_signature__ = recipes-\S+'), zc.buildout.tests.hide_distribute_additions,
'__buildout_signature__ = recipes-SSSSSSSSSSS'), (re.compile('__buildout_signature__ = recipes-\S+'),
(re.compile('[-d] setuptools-\S+[.]egg'), 'setuptools.egg'), '__buildout_signature__ = recipes-SSSSSSSSSSS'),
(re.compile('zc.buildout(-\S+)?[.]egg(-link)?'), (re.compile('[-d] setuptools-\S+[.]egg'), 'setuptools.egg'),
'zc.buildout.egg'), (re.compile('zc.buildout(-\S+)?[.]egg(-link)?'),
(re.compile('creating \S*setup.cfg'), 'creating setup.cfg'), 'zc.buildout.egg'),
(re.compile('hello\%ssetup' % os.path.sep), 'hello/setup'), (re.compile('creating \S*setup.cfg'), 'creating setup.cfg'),
(re.compile('Picked: (\S+) = \S+'), (re.compile('hello\%ssetup' % os.path.sep), 'hello/setup'),
'Picked: \\1 = V.V'), (re.compile('Picked: (\S+) = \S+'),
(re.compile(r'We have a develop egg: zc.buildout (\S+)'), 'Picked: \\1 = V.V'),
'We have a develop egg: zc.buildout X.X.'), (re.compile(r'We have a develop egg: zc.buildout (\S+)'),
(re.compile(r'\\[\\]?'), '/'), 'We have a develop egg: zc.buildout X.X.'),
(re.compile('WindowsError'), 'OSError'), (re.compile(r'\\[\\]?'), '/'),
(re.compile(r'\[Error 17\] Cannot create a file ' (re.compile('WindowsError'), 'OSError'),
r'when that file already exists: '), (re.compile(r'\[Error 17\] Cannot create a file '
'[Errno 17] File exists: ' r'when that file already exists: '),
), '[Errno 17] File exists: '
]) ),
])
), ),
doctest.DocFileSuite( doctest.DocFileSuite(
'testing_bugfix.txt'), 'testing_bugfix.txt',
checker=renormalizing.RENormalizing([
# Python 2.7
(re.compile(
re.escape(
'testrunner.logsupport.NullHandler instance at')),
'testrunner.logsupport.NullHandler object at'),
(re.compile(re.escape('logging.StreamHandler instance at')),
'logging.StreamHandler object at'),
])
),
] ]
# adding bootstrap.txt doctest to the suite # adding bootstrap.txt doctest to the suite
...@@ -3851,17 +4125,38 @@ def test_suite(): ...@@ -3851,17 +4125,38 @@ def test_suite():
if os.path.exists(bootstrap_py): if os.path.exists(bootstrap_py):
test_suite.append(doctest.DocFileSuite( test_suite.append(doctest.DocFileSuite(
'bootstrap.txt', 'bootstrap.txt',
setUp=bootstrapSetup,
tearDown=zc.buildout.testing.buildoutTearDown,
checker=renormalizing.RENormalizing([
zc.buildout.testing.normalize_path,
zc.buildout.testing.normalize_endings,
zc.buildout.testing.normalize_script,
zc.buildout.testing.normalize_egg_py,
normalize_bang,
(re.compile('Downloading.*setuptools.*egg\n'), ''),
(re.compile('options:'), 'Options:'),
(re.compile('usage:'), 'Usage:'),
]),
))
test_suite.append(doctest.DocFileSuite(
'virtualenv.txt',
setUp=easy_install_SetUp, setUp=easy_install_SetUp,
tearDown=zc.buildout.testing.buildoutTearDown, tearDown=zc.buildout.testing.buildoutTearDown,
checker=renormalizing.RENormalizing([ checker=renormalizing.RENormalizing([
zc.buildout.testing.normalize_path, zc.buildout.testing.normalize_path,
zc.buildout.testing.normalize_endings, zc.buildout.testing.normalize_endings,
zc.buildout.testing.normalize_script, zc.buildout.testing.normalize_script,
normalize_bang, zc.buildout.testing.normalize_egg_py,
(re.compile('Downloading.*setuptools.*egg\n'), ''), zc.buildout.tests.hide_distribute_additions,
(re.compile('options:'), 'Options:'), (re.compile('(setuptools|distribute)-\S+-'),
(re.compile('usage:'), 'Usage:'), 'setuptools.egg'),
]), (re.compile('zc.buildout-\S+-'),
'zc.buildout.egg'),
(re.compile(re.escape('#!"/executable_buildout/bin/py"')),
'#!/executable_buildout/bin/py'), # Windows.
(re.compile(re.escape('/broken_s/')),
'/broken_S/'), # Windows.
]),
)) ))
return unittest.TestSuite(test_suite) return unittest.TestSuite(test_suite)
...@@ -3,13 +3,14 @@ Automatic Buildout Updates ...@@ -3,13 +3,14 @@ Automatic Buildout Updates
When a buildout is run, one of the first steps performed is to check When a buildout is run, one of the first steps performed is to check
for updates to either zc.buildout or setuptools. To demonstrate this, for updates to either zc.buildout or setuptools. To demonstrate this,
we've creates some "new releases" of buildout and setuptools in a we've created some "new releases" of buildout and setuptools in a
new_releases folder: new_releases folder:
>>> ls(new_releases) >>> ls(new_releases)
d setuptools d setuptools
- setuptools-99.99-py2.4.egg - setuptools-99.99-py2.4.egg
d zc.buildout d zc.buildout
- zc.buildout-100.0b1-pyN.N.egg
- zc.buildout-99.99-py2.4.egg - zc.buildout-99.99-py2.4.egg
Let's update the sample buildout.cfg to look in this area: Let's update the sample buildout.cfg to look in this area:
...@@ -78,6 +79,13 @@ new versions found in new releases: ...@@ -78,6 +79,13 @@ new versions found in new releases:
zc.buildout 99.99 zc.buildout 99.99
setuptools 99.99 setuptools 99.99
Notice that, even though we have a newer beta version of zc.buildout
available, the final "99.99" was selected. If you want to get non-final
versions, specify a specific version in your buildout's versions
section, you typically want to use the --accept-buildout-test-releases
option to the bootstrap script, which internally uses the
``accept-buildout-test-releases = true`` discussed below.
Our buildout script's site.py has been updated to use the new eggs: Our buildout script's site.py has been updated to use the new eggs:
>>> cat(sample_buildout, 'parts', 'buildout', 'site.py') >>> cat(sample_buildout, 'parts', 'buildout', 'site.py')
...@@ -162,7 +170,7 @@ Or in non-newest mode: ...@@ -162,7 +170,7 @@ Or in non-newest mode:
setuptools 0.6 setuptools 0.6
We also won't upgrade if the buildout script being run isn't in the We also won't upgrade if the buildout script being run isn't in the
buildouts bin directory. To see this we'll create a new buildout buildout's bin directory. To see this we'll create a new buildout
directory: directory:
>>> sample_buildout2 = tmpdir('sample_buildout2') >>> sample_buildout2 = tmpdir('sample_buildout2')
...@@ -187,3 +195,57 @@ directory: ...@@ -187,3 +195,57 @@ directory:
Not upgrading because not running a local buildout command. Not upgrading because not running a local buildout command.
>>> ls('bin') >>> ls('bin')
As mentioned above, the ``accept-buildout-test-releases = true`` means that
newer non-final versions of these dependencies are preferred. Typically
users are not expected to actually manipulate this value. Instead, the
bootstrap script creates a buildout buildout script that passes in the
value as a command line override. This then results in the buildout
script being rewritten to remember the decision.
We'll mimic this by passing the argument actually in the command line.
>>> cd(sample_buildout)
>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... find-links = %(new_releases)s
... index = %(new_releases)s
... parts = show-versions
... develop = showversions
...
... [show-versions]
... recipe = showversions
... """ % dict(new_releases=new_releases))
>>> print system(buildout +
... ' buildout:accept-buildout-test-releases=true'),
... # doctest: +NORMALIZE_WHITESPACE
Getting distribution for 'zc.buildout'.
Got zc.buildout 100.0b1.
Upgraded:
zc.buildout version 100.0b1,
setuptools version 99.99;
restarting.
Generated script '/sample-buildout/bin/buildout'.
NOTE: Accepting early releases of build system packages. Rerun bootstrap
without --accept-buildout-test-releases (-t) to return to default
behavior.
Develop: '/sample-buildout/showversions'
Updating show-versions.
zc.buildout 100.0b1
setuptools 99.99
The buildout script shows the change.
>>> buildout_script = join(sample_buildout, 'bin', 'buildout')
>>> import sys
>>> if sys.platform.startswith('win'):
... buildout_script += '-script.py'
>>> print open(buildout_script).read() # doctest: +ELLIPSIS
#...
sys.argv.insert(1, 'buildout:accept-buildout-test-releases=true')
print ('NOTE: Accepting early releases of build system packages. Rerun '
'bootstrap without --accept-buildout-test-releases (-t) to return to '
'default behavior.')
...
Version 1.5.0 of buildout (and higher) provides the ability to use
buildout directly with a system Python if you use z3c.recipe.scripts or
other isolation-aware recipes that use the sitepackage_safe_scripts function.
Some people use virtualenv to provide similar functionality.
Unfortunately, a problem with the virtualenv executable as of this
writing means that -S will not work properly with it (see
https://bugs.launchpad.net/virtualenv/+bug/572545). This breaks
buildout's approach to providing isolation.
Because of this, if buildout detects an executable with a broken -S
option, it will revert to its pre-1.5.0 behavior. If buildout has been
asked to provide isolation, it will warn the user that isolation will
not be provided by buildout, but proceed. This should give full
backwards compatibility to virtualenv users.
The only minor annoyance in the future may be recipes that explicitly
use the new buildout functionality to provide isolation: as described
above, the builds will proceed, but users will receive warnings that
buildout is not providing isolation itself. The warnings themselves can
be squelched when running bin/buildout with the ``-s`` option or with a
lower verbosity than usual (e.g., one or more ``-q`` options).
For tests, then, we can examine several things. We'll focus on four.
- Running bootstrap with an executable broken in this way will not try to do
any -S tricks.
- Running sitepackage_safe_scripts with a virtualenv will create an
old-style script. This will affect the bin/buildout script that is
created, for instance. If the sitepackage_safe_scripts function is asked
to provide isolation under these circumstances, it will warn that isolation
will not be available, but still create the desired script.
- Using the easy_install Installer or install or build functions and trying
to request isolation will generate a warning and then the isolation request
will be ignored as it proceeds.
- Passing -s (or -q) to the bin/buildout script will squelch warnings.
Testing these involves first creating a Python that exhibits the same
behavior as the problematic one we care about from virtualenv. Let's do that
first.
>>> import os, sys
>>> from zc.buildout.easy_install import _safe_arg
>>> py_path, site_packages_path = make_py()
>>> if sys.platform == 'win32':
... py_script_path = py_path + '-script.py'
... else:
... py_script_path = py_path
...
>>> py_file = open(py_script_path)
>>> py_lines = py_file.readlines()
>>> py_file.close()
>>> py_file = open(py_script_path, 'w')
>>> extra = '''\
... new_argv = argv[:1]
... for ix, val in enumerate(argv[1:]):
... if val.startswith('--'):
... new_argv.append(val)
... if val.startswith('-') and len(val) > 1:
... if 'S' in val:
... val = val.replace('S', '')
... environ['BROKEN_DASH_S'] = 'Y'
... if val != '-':
... new_argv.append(val)
... if 'c' in val:
... new_argv.extend(argv[ix+2:])
... break
... else:
... new_argv.extend(argv[ix+1:])
... argv = new_argv
... '''
>>> for line in py_lines:
... py_file.write(line)
... if line.startswith('environ = os.environ.copy()'):
... py_file.write(extra)
... print 'Rewritten.'
...
Rewritten.
>>> py_file.close()
>>> sitecustomize_path = join(os.path.dirname(site_packages_path),
... 'parts', 'py', 'sitecustomize.py')
>>> sitecustomize_file = open(sitecustomize_path, 'a')
>>> sitecustomize_file.write('''
... import os, sys
... sys.executable = %r
... if 'BROKEN_DASH_S' in os.environ:
... class ImportHook:
... site = None
...
... @classmethod
... def find_module(klass, fullname, path=None):
... if klass.site is None and 'site' in sys.modules:
... # Pop site out of sys.modules. This will be a
... # close-enough approximation of site not being
... # loaded for our tests--it lets us provoke the
... # right errors when the fixes are absent, and
... # works well enough when the fixes are present.
... klass.site = sys.modules.pop('site')
... if fullname == 'ConfigParser':
... raise ImportError(fullname)
... elif fullname == 'site':
... # Keep the site module from being processed twice.
... return klass
...
... @classmethod
... def load_module(klass, fullname):
... if fullname == 'site':
... return klass.site
... raise ImportError(fullname)
...
... sys.meta_path.append(ImportHook)
... ''' % (py_path,))
>>> sitecustomize_file.close()
>>> print call_py(
... _safe_arg(py_path),
... "import ConfigParser")
<BLANKLINE>
>>> print 'X'; print call_py(
... _safe_arg(py_path),
... "import ConfigParser",
... '-S') # doctest: +ELLIPSIS
X...Traceback (most recent call last):
...
ImportError: No module named ConfigParser
<BLANKLINE>
>>> from zc.buildout.easy_install import _has_broken_dash_S
>>> _has_broken_dash_S(py_path)
True
Well, that was ugly, but it seems to have done the trick. The
executable represented by py_path has the same problematic
characteristic as the virtualenv one: -S results in a Python that does
not allow the import of some packages from the standard library. We'll
test with this.
First, let's try running bootstrap.
>>> from os.path import dirname, join
>>> import zc.buildout
>>> bootstrap_py = join(
... dirname(
... dirname(
... dirname(
... dirname(zc.buildout.__file__)
... )
... )
... ),
... 'bootstrap', 'bootstrap.py')
>>> broken_S_buildout = tmpdir('broken_S')
>>> os.chdir(broken_S_buildout)
>>> write('buildout.cfg',
... '''
... [buildout]
... parts =
... ''')
>>> write('bootstrap.py', open(bootstrap_py).read())
>>> print 'X'; print system(
... _safe_arg(py_path)+' '+
... 'bootstrap.py'); print 'X' # doctest: +ELLIPSIS
X...
Generated script '/broken_S/bin/buildout'.
...
If bootstrap didn't look out for a broken -S, that would have failed. Moreover,
take a look at bin/buildout:
>>> cat('bin', 'buildout')
#!/executable_buildout/bin/py
<BLANKLINE>
import sys
sys.path[0:0] = [
'/broken_S/eggs/setuptools-0.0-pyN.N.egg',
'/broken_S/eggs/zc.buildout-0.0-pyN.N.egg',
]
<BLANKLINE>
import zc.buildout.buildout
<BLANKLINE>
if __name__ == '__main__':
zc.buildout.buildout.main()
That's the old-style buildout script: no changes for users with this issue.
Of course, they don't get the new features either, presumably because
they don't need or want them. This means that if they use a recipe that
tries to use a new feature, the behavior needs to degrade gracefully.
Here's an example. We'll switch to another buildout in which it is easier to
use local dev versions of zc.buildout and z3c.recipe.scripts.
>>> os.chdir(dirname(dirname(buildout)))
>>> write('buildout.cfg',
... '''
... [buildout]
... parts = eggs
... find-links = %(link_server)s
...
... [primed_python]
... executable = %(py_path)s
...
... [eggs]
... recipe = z3c.recipe.scripts
... python = primed_python
... interpreter = py
... eggs = demo
... ''' % globals())
>>> print system(buildout) # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS
Installing eggs.
Getting distribution for 'demo'.
Got demo 0.4c1.
Getting distribution for 'demoneeded'.
Got demoneeded 1.2c1.
Generated script '/sample-buildout/bin/demo'.
Generated interpreter '/sample-buildout/bin/py'.
...UserWarning: Buildout has been asked to exclude or limit site-packages
so that builds can be repeatable when using a system Python. However,
the chosen Python executable has a broken implementation of -S (see
https://bugs.launchpad.net/virtualenv/+bug/572545 for an example
problem) and this breaks buildout's ability to isolate site-packages.
If the executable already has a clean site-packages (e.g., using
virtualenv's ``--no-site-packages`` option) you may be getting
equivalent repeatability. To silence this warning, use the -s argument
to the buildout script. Alternatively, use a Python executable with a
working -S (such as a standard Python binary).
warnings.warn(BROKEN_DASH_S_WARNING)
<BLANKLINE>
So, it did what we asked as best it could, but gave a big warning. If
you don't want those warnings for those particular recipes that use the
new features, you can use the "-s" option to squelch the warnings.
>>> print system(buildout + ' -s')
Updating eggs.
<BLANKLINE>
A lower verbosity (one or more -q options) also quiets the warning.
>>> print system(buildout + ' -q')
<BLANKLINE>
Notice that, as we saw before with bin/buildout, the generated scripts
are old-style, because the new-style feature gracefully degrades to the
previous implementation when it encounters an executable with a broken
dash-S.
>>> print 'X'; cat('bin', 'py') # doctest: +ELLIPSIS
X...
<BLANKLINE>
import sys
<BLANKLINE>
sys.path[0:0] = [
'/sample-buildout/eggs/demo-0.4c1-pyN.N.egg',
'/sample-buildout/eggs/demoneeded-1.2c1-pyN.N.egg',
]
...
Change History Change History
************** **************
1.0.0 (unreleased)
==================
(no significant changes)
1.0.0b1 (2010-04-29) 1.0.0b1 (2010-04-29)
==================== ====================
......
include *.txt
recursive-include src *.txt
exclude MANIFEST.in buildout.cfg .bzrignore
...@@ -12,11 +12,9 @@ ...@@ -12,11 +12,9 @@
# #
############################################################################## ##############################################################################
"""Setup for z3c.recipe.scripts package """Setup for z3c.recipe.scripts package
$Id: setup.py 106736 2009-12-18 02:33:08Z gary $
""" """
version = '1.0.0dev' version = '1.0.0'
import os import os
from setuptools import setup, find_packages from setuptools import setup, find_packages
...@@ -60,8 +58,8 @@ setup( ...@@ -60,8 +58,8 @@ setup(
package_dir = {'':'src'}, package_dir = {'':'src'},
namespace_packages = ['z3c', 'z3c.recipe'], namespace_packages = ['z3c', 'z3c.recipe'],
install_requires = [ install_requires = [
'zc.buildout >=1.5.0dev', 'zc.buildout >=1.5.0',
'zc.recipe.egg >=1.2.3dev', 'zc.recipe.egg >=1.3.0',
'setuptools'], 'setuptools'],
tests_require = ['zope.testing'], tests_require = ['zope.testing'],
test_suite = name+'.tests.test_suite', test_suite = name+'.tests.test_suite',
......
...@@ -35,23 +35,13 @@ class Base(ScriptBase): ...@@ -35,23 +35,13 @@ class Base(ScriptBase):
'*') '*')
self.allowed_eggs = tuple(name.strip() for name in value.split('\n')) self.allowed_eggs = tuple(name.strip() for name in value.split('\n'))
value = options.setdefault( self.include_site_packages = options.query_bool(
'include-site-packages', 'include-site-packages',
b_options.get('include-site-packages', 'false')) default=b_options.get('include-site-packages', 'false'))
if value not in ('true', 'false'):
raise zc.buildout.UserError(
"Invalid value for include-site-packages option: %s" %
(value,))
self.include_site_packages = (value == 'true')
value = options.setdefault( self.exec_sitecustomize = options.query_bool(
'exec-sitecustomize', 'exec-sitecustomize',
b_options.get('exec-sitecustomize', 'false')) default=b_options.get('exec-sitecustomize', 'false'))
if value not in ('true', 'false'):
raise zc.buildout.UserError(
"Invalid value for exec-sitecustomize option: %s" %
(value,))
self.exec_sitecustomize = (value == 'true')
class Interpreter(Base): class Interpreter(Base):
......
...@@ -416,24 +416,26 @@ def test_suite(): ...@@ -416,24 +416,26 @@ def test_suite():
'README.txt', 'README.txt',
setUp=setUp, tearDown=zc.buildout.testing.buildoutTearDown, setUp=setUp, tearDown=zc.buildout.testing.buildoutTearDown,
checker=renormalizing.RENormalizing([ checker=renormalizing.RENormalizing([
zc.buildout.testing.normalize_path, zc.buildout.testing.normalize_path,
zc.buildout.testing.normalize_endings, zc.buildout.testing.normalize_endings,
zc.buildout.testing.normalize_script, zc.buildout.testing.normalize_script,
zc.buildout.testing.normalize_egg_py, zc.buildout.testing.normalize_egg_py,
zc.buildout.tests.normalize_bang, zc.buildout.tests.normalize_bang,
(re.compile(r'zc.buildout(-\S+)?[.]egg(-link)?'), zc.buildout.tests.hide_distribute_additions,
'zc.buildout.egg'), zc.buildout.tests.hide_first_index_page_message,
(re.compile('[-d] (setuptools|distribute)-[^-]+-'), 'setuptools-X-'), (re.compile(r'zc.buildout(-\S+)?[.]egg(-link)?'),
(re.compile(r'(setuptools|distribute)-[\w.]+-py'), 'setuptools-X-py'), 'zc.buildout.egg'),
(re.compile(r'eggs\\\\demo'), 'eggs/demo'), (re.compile('[-d] (setuptools|distribute)-[^-]+-'), 'setuptools-X-'),
(re.compile(r'[a-zA-Z]:\\\\foo\\\\bar'), '/foo/bar'), (re.compile(r'(setuptools|distribute)-[\w.]+-py'), 'setuptools-X-py'),
(re.compile(r'\#!\S+\bpython\S*'), '#!/usr/bin/python'), (re.compile(r'eggs\\\\demo'), 'eggs/demo'),
# Normalize generate_script's Windows interpreter to UNIX: (re.compile(r'[a-zA-Z]:\\\\foo\\\\bar'), '/foo/bar'),
(re.compile(r'\nimport subprocess\n'), '\n'), (re.compile(r'\#!\S+\bpython\S*'), '#!/usr/bin/python'),
(re.compile('subprocess\\.call\\(argv, env=environ\\)'), # Normalize generate_script's Windows interpreter to UNIX:
'os.execve(sys.executable, argv, environ)'), (re.compile(r'\nimport subprocess\n'), '\n'),
(re.compile('distribute'), 'setuptools'), (re.compile('subprocess\\.call\\(argv, env=environ\\)'),
]) 'os.execve(sys.executable, argv, environ)'),
(re.compile('distribute'), 'setuptools'),
])
), ),
doctest.DocTestSuite( doctest.DocTestSuite(
setUp=setUp, setUp=setUp,
...@@ -442,6 +444,8 @@ def test_suite(): ...@@ -442,6 +444,8 @@ def test_suite():
zc.buildout.testing.normalize_path, zc.buildout.testing.normalize_path,
zc.buildout.testing.normalize_endings, zc.buildout.testing.normalize_endings,
zc.buildout.testing.normalize_egg_py, zc.buildout.testing.normalize_egg_py,
zc.buildout.tests.hide_distribute_additions,
zc.buildout.tests.hide_first_index_page_message,
(re.compile(r'[a-zA-Z]:\\\\foo\\\\bar'), '/foo/bar'), (re.compile(r'[a-zA-Z]:\\\\foo\\\\bar'), '/foo/bar'),
]), ]),
), ),
......
Change History Change History
************** **************
1.3.0 (unreleased)
==================
- Small further refactorings past 1.2.3b1 to be compatible with zc.buildout
1.5.0.
1.2.3b1 (2010-04-29) 1.2.3b1 (2010-04-29)
==================== ====================
......
include *.txt
recursive-include src *.txt
exclude MANIFEST.in buildout.cfg .bzrignore
...@@ -12,11 +12,9 @@ ...@@ -12,11 +12,9 @@
# #
############################################################################## ##############################################################################
"""Setup for zc.recipe.egg package """Setup for zc.recipe.egg package
$Id$
""" """
version = '1.2.3dev' version = '1.3.0'
import os import os
from setuptools import setup, find_packages from setuptools import setup, find_packages
...@@ -66,7 +64,7 @@ setup( ...@@ -66,7 +64,7 @@ setup(
package_dir = {'':'src'}, package_dir = {'':'src'},
namespace_packages = ['zc', 'zc.recipe'], namespace_packages = ['zc', 'zc.recipe'],
install_requires = [ install_requires = [
'zc.buildout >=1.5.0dev', 'zc.buildout >=1.5.0',
'setuptools'], 'setuptools'],
tests_require = ['zope.testing'], tests_require = ['zope.testing'],
test_suite = name+'.tests.test_suite', test_suite = name+'.tests.test_suite',
......
...@@ -52,9 +52,6 @@ class Eggs(object): ...@@ -52,9 +52,6 @@ class Eggs(object):
options['develop-eggs-directory'] = b_options['develop-eggs-directory'] options['develop-eggs-directory'] = b_options['develop-eggs-directory']
options['_d'] = options['develop-eggs-directory'] # backward compat. options['_d'] = options['develop-eggs-directory'] # backward compat.
# verify that this is None, 'true' or 'false'
get_bool(options, 'unzip')
python = options.setdefault('python', b_options['python']) python = options.setdefault('python', b_options['python'])
options['executable'] = buildout[python]['executable'] options['executable'] = buildout[python]['executable']
...@@ -84,7 +81,7 @@ class Eggs(object): ...@@ -84,7 +81,7 @@ class Eggs(object):
else: else:
kw = {} kw = {}
if 'unzip' in options: if 'unzip' in options:
kw['always_unzip'] = get_bool(options, 'unzip') kw['always_unzip'] = options.query_bool('unzip', None)
ws = zc.buildout.easy_install.install( ws = zc.buildout.easy_install.install(
distributions, options['eggs-directory'], distributions, options['eggs-directory'],
links=self.links, links=self.links,
...@@ -159,7 +156,7 @@ class ScriptBase(Eggs): ...@@ -159,7 +156,7 @@ class ScriptBase(Eggs):
raise zc.buildout.UserError("Invalid entry point") raise zc.buildout.UserError("Invalid entry point")
reqs.append(parsed.groups()) reqs.append(parsed.groups())
if get_bool(options, 'dependent-scripts'): if options.query_bool('dependent-scripts', 'false'):
# Generate scripts for all packages in the working set, # Generate scripts for all packages in the working set,
# except setuptools. # except setuptools.
reqs = list(reqs) reqs = list(reqs)
...@@ -192,17 +189,4 @@ class Scripts(ScriptBase): ...@@ -192,17 +189,4 @@ class Scripts(ScriptBase):
relative_paths=self._relative_paths relative_paths=self._relative_paths
) )
def get_bool(options, name, default=False):
value = options.get(name)
if not value:
return default
if value == 'true':
return True
elif value == 'false':
return False
else:
raise zc.buildout.UserError(
"Invalid value for %s option: %s" % (name, value))
Egg = Scripts Egg = Scripts
...@@ -43,54 +43,58 @@ def test_suite(): ...@@ -43,54 +43,58 @@ def test_suite():
'README.txt', 'README.txt',
setUp=setUp, tearDown=zc.buildout.testing.buildoutTearDown, setUp=setUp, tearDown=zc.buildout.testing.buildoutTearDown,
checker=renormalizing.RENormalizing([ checker=renormalizing.RENormalizing([
zc.buildout.testing.normalize_path, zc.buildout.testing.normalize_path,
zc.buildout.testing.normalize_endings, zc.buildout.testing.normalize_endings,
zc.buildout.testing.normalize_script, zc.buildout.testing.normalize_script,
zc.buildout.testing.normalize_egg_py, zc.buildout.testing.normalize_egg_py,
zc.buildout.tests.normalize_bang, zc.buildout.tests.normalize_bang,
(re.compile('zc.buildout(-\S+)?[.]egg(-link)?'), zc.buildout.tests.hide_distribute_additions,
'zc.buildout.egg'), (re.compile('zc.buildout(-\S+)?[.]egg(-link)?'),
(re.compile('[-d] (setuptools|distribute)-[^-]+-'), 'zc.buildout.egg'),
'setuptools-X-'), (re.compile('[-d] (setuptools|distribute)-[^-]+-'),
(re.compile(r'eggs\\\\demo'), 'eggs/demo'), 'setuptools-X-'),
(re.compile(r'[a-zA-Z]:\\\\foo\\\\bar'), '/foo/bar'), (re.compile(r'eggs\\\\demo'), 'eggs/demo'),
# Distribute unzips eggs by default. (re.compile(r'[a-zA-Z]:\\\\foo\\\\bar'), '/foo/bar'),
(re.compile('\- demoneeded'), 'd demoneeded'), # Distribute unzips eggs by default.
]) (re.compile('\- demoneeded'), 'd demoneeded'),
])
), ),
doctest.DocFileSuite( doctest.DocFileSuite(
'api.txt', 'api.txt',
setUp=setUp, tearDown=zc.buildout.testing.buildoutTearDown, setUp=setUp, tearDown=zc.buildout.testing.buildoutTearDown,
checker=renormalizing.RENormalizing([ checker=renormalizing.RENormalizing([
zc.buildout.testing.normalize_path, zc.buildout.testing.normalize_path,
zc.buildout.testing.normalize_endings, zc.buildout.testing.normalize_endings,
(re.compile('__buildout_signature__ = ' zc.buildout.tests.hide_distribute_additions,
'sample-\S+\s+' (re.compile('__buildout_signature__ = '
'zc.recipe.egg-\S+\s+' 'sample-\S+\s+'
'(setuptools|distribute)-\S+\s+' 'zc.recipe.egg-\S+\s+'
'zc.buildout-\S+\s*' '(setuptools|distribute)-\S+\s+'
), 'zc.buildout-\S+\s*'
'__buildout_signature__ = sample- zc.recipe.egg-\n'), ),
(re.compile('executable = [\S ]+python\S*', re.I), '__buildout_signature__ = sample- zc.recipe.egg-\n'),
'executable = python'), (re.compile('executable = [\S ]+python\S*', re.I),
(re.compile('find-links = http://localhost:\d+/'), 'executable = python'),
'find-links = http://localhost:8080/'), (re.compile('find-links = http://localhost:\d+/'),
(re.compile('index = http://localhost:\d+/index'), 'find-links = http://localhost:8080/'),
'index = http://localhost:8080/index'), (re.compile('index = http://localhost:\d+/index'),
]) 'index = http://localhost:8080/index'),
])
), ),
doctest.DocFileSuite( doctest.DocFileSuite(
'custom.txt', 'custom.txt',
setUp=setUp, tearDown=zc.buildout.testing.buildoutTearDown, setUp=setUp, tearDown=zc.buildout.testing.buildoutTearDown,
checker=renormalizing.RENormalizing([ checker=renormalizing.RENormalizing([
zc.buildout.testing.normalize_path, zc.buildout.testing.normalize_path,
zc.buildout.testing.normalize_endings, zc.buildout.testing.normalize_endings,
(re.compile("(d ((ext)?demo(needed)?|other)" zc.buildout.tests.hide_distribute_additions,
"-\d[.]\d-py)\d[.]\d(-\S+)?[.]egg"), zc.buildout.tests.hide_zip_safe_message,
'\\1V.V.egg'), (re.compile("(d ((ext)?demo(needed)?|other)"
(re.compile('extdemo.c\n.+\\extdemo.exp\n'), ''), "-\d[.]\d-py)\d[.]\d(-\S+)?[.]egg"),
(re.compile('extdemo[.]pyd'), 'extdemo.so') '\\1V.V.egg'),
]), (re.compile('extdemo.c\n.+\\extdemo.exp\n'), ''),
(re.compile('extdemo[.]pyd'), 'extdemo.so')
]),
), ),
)) ))
...@@ -104,21 +108,22 @@ def test_suite(): ...@@ -104,21 +108,22 @@ def test_suite():
setUp=setUpSelecting, setUp=setUpSelecting,
tearDown=zc.buildout.testing.buildoutTearDown, tearDown=zc.buildout.testing.buildoutTearDown,
checker=renormalizing.RENormalizing([ checker=renormalizing.RENormalizing([
zc.buildout.testing.normalize_path, zc.buildout.testing.normalize_path,
zc.buildout.testing.normalize_endings, zc.buildout.testing.normalize_endings,
zc.buildout.testing.normalize_script, zc.buildout.testing.normalize_script,
(re.compile('Got (setuptools|distribute) \S+'), zc.buildout.tests.hide_distribute_additions,
'Got setuptools V'), (re.compile('Got (setuptools|distribute) \S+'),
(re.compile('([d-] )?(setuptools|distribute)-\S+-py'), 'Got setuptools V'),
'setuptools-V-py'), (re.compile('([d-] )?(setuptools|distribute)-\S+-py'),
(re.compile('-py2[.][0-35-9][.]'), 'py2.5.'), 'setuptools-V-py'),
(re.compile('zc.buildout-\S+[.]egg'), (re.compile('-py2[.][0-35-9][.]'), 'py2.5.'),
'zc.buildout.egg'), (re.compile('zc.buildout-\S+[.]egg'),
(re.compile('zc.buildout[.]egg-link'), 'zc.buildout.egg'),
'zc.buildout.egg'), (re.compile('zc.buildout[.]egg-link'),
# Distribute unzips eggs by default. 'zc.buildout.egg'),
(re.compile('\- demoneeded'), 'd demoneeded'), # Distribute unzips eggs by default.
]), (re.compile('\- demoneeded'), 'd demoneeded'),
]),
), ),
) )
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment