Commit 23fe23a4 authored by Gary Poster's avatar Gary Poster

merge lp:~gary/zc.buildout/python-support (also available as...

merge lp:~gary/zc.buildout/python-support (also available as svn+ssh://svn.zope.org/repos/main/zc.buildout/branches/gary-launchpad).  This adds support for system Pythons, as well as other changes, such as improved Distribute support and some cleanups.
parents a032f8e2 95719f46
.installed.cfg
bin
build
develop-eggs
eggs
parts
src/zc.buildout.egg-info
z3c.recipe.scripts_/src/z3c.recipe.scripts.egg-info
zc.recipe.egg_/src/zc.recipe.egg.egg-info
Change History Change History
************** **************
1.4.4 (?) 1.5.0 (201?-??-??)
========= ==================
New Features:
- Added buildout:socket-timout option so that socket timeout can be configured
both from command line and from config files. (gotcha)
- Buildout can be safely used with a system Python (or any Python with code
in site-packages), as long as you use the new z3c.recipe.scripts
recipe to generate scripts and interpreters, rather than zc.recipe.egg.
zc.recipe.egg is still a fully supported, and simpler, way of
generating scripts and interpreters if you are using a "clean" Python,
without code installed in site-packages. It keeps its previous behavior in
order to provide backwards compatibility.
The z3c.recipe.scripts recipe allows you to control how you use the
code in site-packages. You can exclude it entirely; allow eggs in it
to fulfill package dependencies declared in setup.py and buildout
configuration; allow it to be available but not used to fulfill
dependencies declared in setup.py or buildout configuration; or only
allow certain eggs in site-packages to fulfill dependencies.
- Added new function, ``zc.buildout.easy_install.sitepackage_safe_scripts``,
to generate scripts and interpreter. It produces a full-featured
interpreter (all command-line options supported) and the ability to
safely let scripts include site packages, such as with a system
Python. The ``z3c.recipe.scripts`` recipe uses this new function.
- Improve bootstrap.
* New options let you specify where to find ez_setup.py and where to find
a download cache. These options can keep bootstrap from going over the
network.
* Another new option lets you specify where to put generated eggs.
* The buildout script generated by bootstrap honors more of the settings
in the designated configuration file (e.g., buildout.cfg).
- You can develop zc.buildout using Distribute instead of Setuptools. Use
the --distribute option on the dev.py script. (Releases should be tested
with both Distribute and Setuptools.)
- The ``distribute-version`` now works in the [buildout] section, mirroring
the ``setuptools-version`` option (this is for consistency; using the
general-purpose ``versions`` option is preferred).
Bugs fixed:
- Using Distribute with the ``allow-picked-versions = false`` buildout
option no longer causes an error.
- The handling and documenting of default buildout options was normalized.
This means, among other things, that ``bin/buildout -vv`` and
``bin/buildout annotate`` correctly list more of the options.
- Installing a namespace package using a Python that already has a package
in the same namespace (e.g., in the Python's site-packages) failed in
some cases. It is now handled correctly.
- Another variation of this error showed itself when at least two
dependencies were in a shared location like site-packages, and the
first one met the "versions" setting. The first dependency would be
added, but subsequent dependencies from the same location (e.g.,
site-packages) would use the version of the package found in the
shared location, ignoring the version setting. This is also now
handled correctly.
1.4.3 (2009-12-10) 1.4.3 (2009-12-10)
================== ==================
......
...@@ -35,7 +35,15 @@ Existing recipes include: ...@@ -35,7 +35,15 @@ Existing recipes include:
`zc.recipe.egg <http://pypi.python.org/pypi/zc.recipe.egg>`_ `zc.recipe.egg <http://pypi.python.org/pypi/zc.recipe.egg>`_
The egg recipe installes one or more eggs, with their The egg recipe installes one or more eggs, with their
dependencies. It installs their console-script entry points with dependencies. It installs their console-script entry points with
the needed eggs included in their paths. the needed eggs included in their paths. It is suitable for use with
a "clean" Python: one without packages installed in site-packages.
`z3c.recipe.scripts <http://pypi.python.org/pypi/z3c.recipe.scripts>`_
Like zc.recipe.egg, this recipe builds interpreter scripts and entry
point scripts based on eggs. It can be used with a Python that has
packages installed in site-packages, such as a system Python. The
interpreter also has more features than the one offered by
zc.recipe.egg.
`zc.recipe.testrunner <http://pypi.python.org/pypi/zc.recipe.testrunner>`_ `zc.recipe.testrunner <http://pypi.python.org/pypi/zc.recipe.testrunner>`_
The testrunner egg creates a test runner script for one or The testrunner egg creates a test runner script for one or
......
...@@ -20,102 +20,187 @@ use the -c option to specify an alternate configuration file. ...@@ -20,102 +20,187 @@ use the -c option to specify an alternate configuration file.
$Id$ $Id$
""" """
import os, shutil, sys, tempfile, urllib2 import os, shutil, sys, tempfile, textwrap, urllib, urllib2
from optparse import OptionParser from optparse import OptionParser
tmpeggs = tempfile.mkdtemp() if sys.platform == 'win32':
def quote(c):
if ' ' in c:
return '"%s"' % c # work around spawn lamosity on windows
else:
return c
else:
quote = str
# In order to be more robust in the face of system Pythons, we want to
# run without site-packages loaded. This is somewhat tricky, in
# particular because Python 2.6's distutils imports site, so starting
# with the -S flag is not sufficient. However, we'll start with that:
if 'site' in sys.modules:
# We will restart with python -S.
args = sys.argv[:]
args[0:0] = [sys.executable, '-S']
args = map(quote, args)
os.execv(sys.executable, args)
# Now we are running with -S. We'll get the clean sys.path, import site
# because distutils will do it later, and then reset the path and clean
# out any namespace packages from site-packages that might have been
# loaded by .pth files.
clean_path = sys.path[:]
import site
sys.path[:] = clean_path
for k, v in sys.modules.items():
if (hasattr(v, '__path__') and
len(v.__path__)==1 and
not os.path.exists(os.path.join(v.__path__[0],'__init__.py'))):
# This is a namespace package. Remove it.
sys.modules.pop(k)
is_jython = sys.platform.startswith('java') is_jython = sys.platform.startswith('java')
setuptools_source = 'http://peak.telecommunity.com/dist/ez_setup.py'
distribute_source = 'http://python-distribute.org/distribute_setup.py'
# parsing arguments # parsing arguments
parser = OptionParser() def normalize_to_url(option, opt_str, value, parser):
if value:
if '://' not in value: # It doesn't smell like a URL.
value = 'file://%s' % (
urllib.pathname2url(
os.path.abspath(os.path.expanduser(value))),)
if opt_str == '--download-base' and not value.endswith('/'):
# Download base needs a trailing slash to make the world happy.
value += '/'
else:
value = None
name = opt_str[2:].replace('-', '_')
setattr(parser.values, name, value)
usage = '''\
[DESIRED PYTHON FOR BUILDOUT] bootstrap.py [options]
Bootstraps a buildout-based project.
Simply run this script in a directory containing a buildout.cfg, using the
Python that you want bin/buildout to use.
Note that by using --setup-source and --download-base to point to
local resources, you can keep this script from going over the network.
'''
parser = OptionParser(usage=usage)
parser.add_option("-v", "--version", dest="version", parser.add_option("-v", "--version", dest="version",
help="use a specific zc.buildout version") help="use a specific zc.buildout version")
parser.add_option("-d", "--distribute", parser.add_option("-d", "--distribute",
action="store_true", dest="distribute", default=False, action="store_true", dest="use_distribute", default=False,
help="Use Distribute rather than Setuptools.") help="Use Distribute rather than Setuptools.")
parser.add_option("--setup-source", action="callback", dest="setup_source",
callback=normalize_to_url, nargs=1, type="string",
help=("Specify a URL or file location for the setup file. "
"If you use Setuptools, this will default to " +
setuptools_source + "; if you use Distribute, this "
"will default to " + distribute_source +"."))
parser.add_option("--download-base", action="callback", dest="download_base",
callback=normalize_to_url, nargs=1, type="string",
help=("Specify a URL or directory for downloading "
"zc.buildout and either Setuptools or Distribute. "
"Defaults to PyPI."))
parser.add_option("--eggs",
help=("Specify a directory for storing eggs. Defaults to "
"a temporary directory that is deleted when the "
"bootstrap script completes."))
parser.add_option("-c", None, action="store", dest="config_file", parser.add_option("-c", None, action="store", dest="config_file",
help=("Specify the path to the buildout configuration " help=("Specify the path to the buildout configuration "
"file to be used.")) "file to be used."))
options, args = parser.parse_args() options, args = parser.parse_args()
# if -c was provided, we push it back into args for buildout' main function # if -c was provided, we push it back into args for buildout's main function
if options.config_file is not None: if options.config_file is not None:
args += ['-c', options.config_file] args += ['-c', options.config_file]
if options.version is not None: if options.eggs:
VERSION = '==%s' % options.version eggs_dir = os.path.abspath(os.path.expanduser(options.eggs))
else: else:
VERSION = '' eggs_dir = tempfile.mkdtemp()
if options.setup_source is None:
if options.use_distribute:
options.setup_source = distribute_source
else:
options.setup_source = setuptools_source
USE_DISTRIBUTE = options.distribute
args = args + ['bootstrap'] args = args + ['bootstrap']
to_reload = False
try: try:
to_reload = False
import pkg_resources import pkg_resources
to_reload = True
if not hasattr(pkg_resources, '_distribute'): if not hasattr(pkg_resources, '_distribute'):
to_reload = True
raise ImportError raise ImportError
import setuptools # A flag. Sometimes pkg_resources is installed alone.
except ImportError: except ImportError:
ez_code = urllib2.urlopen(
options.setup_source).read().replace('\r\n', '\n')
ez = {} ez = {}
if USE_DISTRIBUTE: exec ez_code in ez
exec urllib2.urlopen('http://python-distribute.org/distribute_setup.py' setup_args = dict(to_dir=eggs_dir, download_delay=0)
).read() in ez if options.download_base:
ez['use_setuptools'](to_dir=tmpeggs, download_delay=0, no_fake=True) setup_args['download_base'] = options.download_base
else: if options.use_distribute:
exec urllib2.urlopen('http://peak.telecommunity.com/dist/ez_setup.py' setup_args['no_fake'] = True
).read() in ez ez['use_setuptools'](**setup_args)
ez['use_setuptools'](to_dir=tmpeggs, download_delay=0)
if to_reload: if to_reload:
reload(pkg_resources) reload(pkg_resources)
else: else:
import pkg_resources import pkg_resources
# This does not (always?) update the default working set. We will
if sys.platform == 'win32': # do it.
def quote(c): for path in sys.path:
if ' ' in c: if path not in pkg_resources.working_set.entries:
return '"%s"' % c # work around spawn lamosity on windows pkg_resources.working_set.add_entry(path)
else:
return c cmd = [quote(sys.executable),
else: '-c',
def quote (c): quote('from setuptools.command.easy_install import main; main()'),
return c '-mqNxd',
quote(eggs_dir)]
cmd = 'from setuptools.command.easy_install import main; main()'
ws = pkg_resources.working_set if options.download_base:
cmd.extend(['-f', quote(options.download_base)])
if USE_DISTRIBUTE:
requirement = 'distribute' requirement = 'zc.buildout'
if options.version:
requirement = '=='.join((requirement, options.version))
cmd.append(requirement)
if options.use_distribute:
setup_requirement = 'distribute'
else: else:
requirement = 'setuptools' setup_requirement = 'setuptools'
ws = pkg_resources.working_set
env = dict(
os.environ,
PYTHONPATH=ws.find(
pkg_resources.Requirement.parse(setup_requirement)).location)
if is_jython: if is_jython:
import subprocess import subprocess
exitcode = subprocess.Popen(cmd, env=env).wait()
assert subprocess.Popen([sys.executable] + ['-c', quote(cmd), '-mqNxd', else: # Windows prefers this, apparently; otherwise we would prefer subprocess
quote(tmpeggs), 'zc.buildout' + VERSION], exitcode = os.spawnle(*([os.P_WAIT, sys.executable] + cmd + [env]))
env=dict(os.environ, if exitcode != 0:
PYTHONPATH= sys.stdout.flush()
ws.find(pkg_resources.Requirement.parse(requirement)).location sys.stderr.flush()
), print ("An error occured when trying to install zc.buildout. "
).wait() == 0 "Look above this message for any errors that "
"were output by easy_install.")
else: sys.exit(exitcode)
assert os.spawnle(
os.P_WAIT, sys.executable, quote (sys.executable), ws.add_entry(eggs_dir)
'-c', quote (cmd), '-mqNxd', quote (tmpeggs), 'zc.buildout' + VERSION, ws.require(requirement)
dict(os.environ,
PYTHONPATH=
ws.find(pkg_resources.Requirement.parse(requirement)).location
),
) == 0
ws.add_entry(tmpeggs)
ws.require('zc.buildout' + VERSION)
import zc.buildout.buildout import zc.buildout.buildout
zc.buildout.buildout.main(args) zc.buildout.buildout.main(args)
shutil.rmtree(tmpeggs) if not options.eggs: # clean up temporary egg directory
shutil.rmtree(eggs_dir)
[buildout] [buildout]
develop = zc.recipe.egg_ . develop = zc.recipe.egg_ z3c.recipe.scripts_ .
parts = test oltest py parts = test oltest py
[py] [py]
recipe = zc.recipe.egg recipe = z3c.recipe.scripts
eggs = zc.buildout eggs = zc.buildout
zope.testing zope.testing
interpreter = py interpreter = py
[test] [test]
recipe = zc.recipe.testrunner recipe = zc.recipe.testrunner
eggs = eggs =
zc.buildout zc.buildout
zc.recipe.egg zc.recipe.egg
z3c.recipe.scripts
# Tests that can be run wo a network # Tests that can be run wo a network
[oltest] [oltest]
recipe = zc.recipe.testrunner recipe = zc.recipe.testrunner
eggs = eggs =
zc.buildout zc.buildout
zc.recipe.egg zc.recipe.egg
z3c.recipe.scripts
defaults = defaults =
[ [
'-t', '-t',
......
...@@ -13,43 +13,126 @@ ...@@ -13,43 +13,126 @@
############################################################################## ##############################################################################
"""Bootstrap the buildout project itself. """Bootstrap the buildout project itself.
This is different from a normal boostrapping process because the This is different from a normal bootstrapping process because the
buildout egg itself is installed as a develop egg. buildout egg itself is installed as a develop egg.
$Id$ $Id$
""" """
import os, shutil, sys, subprocess, urllib2 import os, shutil, sys, subprocess, urllib2
from optparse import OptionParser
if sys.platform == 'win32':
def quote(c):
if ' ' in c:
return '"%s"' % c # work around spawn lamosity on windows
else:
return c
else:
quote = str
# In order to be more robust in the face of system Pythons, we want to
# run without site-packages loaded. This is somewhat tricky, in
# particular because Python 2.6's distutils imports site, so starting
# with the -S flag is not sufficient. However, we'll start with that:
if 'site' in sys.modules:
# We will restart with python -S.
args = sys.argv[:]
args[0:0] = [sys.executable, '-S']
args = map(quote, args)
os.execv(sys.executable, args)
# Now we are running with -S. We'll get the clean sys.path, import site
# because distutils will do it later, and then reset the path and clean
# out any namespace packages from site-packages that might have been
# loaded by .pth files.
clean_path = sys.path[:]
import site
sys.path[:] = clean_path
for k, v in sys.modules.items():
if (hasattr(v, '__path__') and
len(v.__path__)==1 and
not os.path.exists(os.path.join(v.__path__[0],'__init__.py'))):
# This is a namespace package. Remove it.
sys.modules.pop(k)
is_jython = sys.platform.startswith('java') is_jython = sys.platform.startswith('java')
setuptools_source = 'http://peak.telecommunity.com/dist/ez_setup.py'
distribute_source = 'http://python-distribute.org/distribute_setup.py'
usage = '''\
[DESIRED PYTHON FOR DEVELOPING BUILDOUT] dev.py [options]
Bootstraps buildout itself for development.
This is different from a normal bootstrapping process because the
buildout egg itself is installed as a develop egg.
'''
parser = OptionParser(usage=usage)
parser.add_option("-d", "--distribute",
action="store_true", dest="use_distribute", default=False,
help="Use Distribute rather than Setuptools.")
options, args = parser.parse_args()
if args:
parser.error('This script accepts no arguments other than its options.')
if options.use_distribute:
setup_source = distribute_source
else:
setup_source = setuptools_source
for d in 'eggs', 'develop-eggs', 'bin': for d in 'eggs', 'develop-eggs', 'bin':
if not os.path.exists(d): if not os.path.exists(d):
os.mkdir(d) os.mkdir(d)
if os.path.isdir('build'): if os.path.isdir('build'):
shutil.rmtree('build') shutil.rmtree('build')
try: try:
to_reload = False
import pkg_resources import pkg_resources
to_reload = True
if not hasattr(pkg_resources, '_distribute'):
raise ImportError
import setuptools # A flag. Sometimes pkg_resources is installed alone.
except ImportError: except ImportError:
ez_code = urllib2.urlopen(setup_source).read().replace('\r\n', '\n')
ez = {} ez = {}
exec urllib2.urlopen('http://peak.telecommunity.com/dist/ez_setup.py' exec ez_code in ez
).read() in ez setup_args = dict(to_dir='eggs', download_delay=0)
ez['use_setuptools'](to_dir='eggs', download_delay=0) if options.use_distribute:
setup_args['no_fake'] = True
import pkg_resources ez['use_setuptools'](**setup_args)
if to_reload:
reload(pkg_resources)
else:
import pkg_resources
# This does not (always?) update the default working set. We will
# do it.
for path in sys.path:
if path not in pkg_resources.working_set.entries:
pkg_resources.working_set.add_entry(path)
env = os.environ.copy() # Windows needs yet-to-be-determined values from this.
env['PYTHONPATH'] = os.path.dirname(pkg_resources.__file__)
subprocess.Popen( subprocess.Popen(
[sys.executable] + [sys.executable] +
['setup.py', '-q', 'develop', '-m', '-x', '-d', 'develop-eggs'], ['setup.py', '-q', 'develop', '-m', '-x', '-d', 'develop-eggs'],
env = {'PYTHONPATH': os.path.dirname(pkg_resources.__file__)}).wait() env=env).wait()
pkg_resources.working_set.add_entry('src') pkg_resources.working_set.add_entry('src')
import zc.buildout.easy_install import zc.buildout.easy_install
zc.buildout.easy_install.scripts( if not os.path.exists('parts'):
['zc.buildout'], pkg_resources.working_set , sys.executable, 'bin') os.mkdir('parts')
partsdir = os.path.join('parts', 'buildout')
if not os.path.exists(partsdir):
os.mkdir(partsdir)
zc.buildout.easy_install.sitepackage_safe_scripts(
'bin', pkg_resources.working_set, sys.executable, partsdir,
reqs=['zc.buildout'])
bin_buildout = os.path.join('bin', 'buildout') bin_buildout = os.path.join('bin', 'buildout')
...@@ -57,4 +140,5 @@ if is_jython: ...@@ -57,4 +140,5 @@ if is_jython:
# Jython needs the script to be called twice via sys.executable # Jython needs the script to be called twice via sys.executable
assert subprocess.Popen([sys.executable] + [bin_buildout]).wait() == 0 assert subprocess.Popen([sys.executable] + [bin_buildout]).wait() == 0
sys.exit(subprocess.Popen(bin_buildout).wait()) sys.exit(subprocess.Popen(bin_buildout).wait())
...@@ -12,7 +12,7 @@ ...@@ -12,7 +12,7 @@
# #
############################################################################## ##############################################################################
name = "zc.buildout" name = "zc.buildout"
version = "1.4.4dev" version = "1.5.0dev"
import os import os
from setuptools import setup from setuptools import setup
......
...@@ -47,7 +47,7 @@ Make sure the bootstrap script actually works:: ...@@ -47,7 +47,7 @@ Make sure the bootstrap script actually works::
X... X...
d zc.buildout-...egg d zc.buildout-...egg
Now trying the `--version` option, that let you define a version for Now we will try the `--version` option, which lets you define a version for
`zc.buildout`. If not provided, bootstrap will look for the latest one. `zc.buildout`. If not provided, bootstrap will look for the latest one.
Let's try with an unknown version:: Let's try with an unknown version::
...@@ -57,7 +57,7 @@ Let's try with an unknown version:: ...@@ -57,7 +57,7 @@ Let's try with an unknown version::
... 'bootstrap.py --version UNKNOWN'); print 'X' # doctest: +ELLIPSIS ... 'bootstrap.py --version UNKNOWN'); print 'X' # doctest: +ELLIPSIS
... ...
X X
No local packages or download links found for zc.buildout==UNKNOWN No local packages or download links found for zc.buildout==UNKNOWN...
... ...
Now let's try with `1.1.2`, which happens to exist:: Now let's try with `1.1.2`, which happens to exist::
...@@ -119,8 +119,8 @@ Let's make sure the generated `buildout` script uses it:: ...@@ -119,8 +119,8 @@ Let's make sure the generated `buildout` script uses it::
zc.buildout.buildout.main() zc.buildout.buildout.main()
<BLANKLINE> <BLANKLINE>
`zc.buildout` now can also run with `Distribute` with the `--distribute` option:: `zc.buildout` now can also run with `Distribute` with the `--distribute`
option::
>>> print 'X'; print system( >>> print 'X'; print system(
... zc.buildout.easy_install._safe_arg(sys.executable)+' '+ ... zc.buildout.easy_install._safe_arg(sys.executable)+' '+
...@@ -153,7 +153,8 @@ Make sure both options can be used together:: ...@@ -153,7 +153,8 @@ Make sure both options can be used together::
>>> print 'X'; print system( >>> print 'X'; print system(
... zc.buildout.easy_install._safe_arg(sys.executable)+' '+ ... zc.buildout.easy_install._safe_arg(sys.executable)+' '+
... 'bootstrap.py --distribute --version 1.2.1'); print 'X' # doctest: +ELLIPSIS ... 'bootstrap.py --distribute --version 1.2.1'); print 'X'
... # doctest: +ELLIPSIS
... ...
X X
... ...
...@@ -161,7 +162,8 @@ Make sure both options can be used together:: ...@@ -161,7 +162,8 @@ Make sure both options can be used together::
<BLANKLINE> <BLANKLINE>
X X
Let's make sure the generated `buildout` script uses ``Distribute`` *and* ``zc.buildout-1.2.1``:: Let's make sure the generated `buildout` script uses ``Distribute`` *and*
``zc.buildout-1.2.1``::
>>> print open(buildout_script).read() # doctest: +ELLIPSIS >>> print open(buildout_script).read() # doctest: +ELLIPSIS
#... #...
...@@ -194,4 +196,70 @@ Last, the -c option needs to work on bootstrap.py:: ...@@ -194,4 +196,70 @@ Last, the -c option needs to work on bootstrap.py::
<BLANKLINE> <BLANKLINE>
X X
You can specify a location of ez_setup.py or distribute_setup, so you
can rely on a local or remote location. We'll write our own ez_setup.py
that we will also use to test some other bootstrap options.
>>> write('ez_setup.py', '''\
... def use_setuptools(**kwargs):
... import sys, pprint
... pprint.pprint(kwargs, width=40)
... sys.exit()
... ''')
>>> print system(
... zc.buildout.easy_install._safe_arg(sys.executable)+' '+
... 'bootstrap.py --setup-source=./ez_setup.py')
... # doctest: +ELLIPSIS
{'download_delay': 0,
'to_dir': '...'}
<BLANKLINE>
You can also pass a download-cache, and a place in which eggs should be stored
(they are normally stored in a temporary directory).
>>> print system(
... zc.buildout.easy_install._safe_arg(sys.executable)+' '+
... 'bootstrap.py --setup-source=./ez_setup.py '+
... '--download-base=./download-cache --eggs=eggs')
... # doctest: +ELLIPSIS
{'download_base': '/sample/download-cache/',
'download_delay': 0,
'to_dir': '/sample/eggs'}
<BLANKLINE>
Here's the entire help text.
>>> print system(
... zc.buildout.easy_install._safe_arg(sys.executable)+' '+
... 'bootstrap.py --help'),
... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
Usage: [DESIRED PYTHON FOR BUILDOUT] bootstrap.py [options]
<BLANKLINE>
Bootstraps a buildout-based project.
<BLANKLINE>
Simply run this script in a directory containing a buildout.cfg, using the
Python that you want bin/buildout to use.
<BLANKLINE>
Note that by using --setup-source and --download-base to point to
local resources, you can keep this script from going over the network.
<BLANKLINE>
<BLANKLINE>
Options:
-h, --help show this help message and exit
-v VERSION, --version=VERSION
use a specific zc.buildout version
-d, --distribute Use Distribute rather than Setuptools.
--setup-source=SETUP_SOURCE
Specify a URL or file location for the setup file. If
you use Setuptools, this will default to
http://peak.telecommunity.com/dist/ez_setup.py; if you
use Distribute, this will default to http://python-
distribute.org/distribute_setup.py.
--download-base=DOWNLOAD_BASE
Specify a URL or directory for downloading zc.buildout
and either Setuptools or Distribute. Defaults to PyPI.
--eggs=EGGS Specify a directory for storing eggs. Defaults to a
temporary directory that is deleted when the bootstrap
script completes.
-c CONFIG_FILE Specify the path to the buildout configuration file to
be used.
This diff is collapsed.
...@@ -56,10 +56,9 @@ The bin directory contains scripts. ...@@ -56,10 +56,9 @@ The bin directory contains scripts.
- setuptools-0.6-py2.4.egg - setuptools-0.6-py2.4.egg
- zc.buildout-1.0-py2.4.egg - zc.buildout-1.0-py2.4.egg
The develop-eggs and parts directories are initially empty: The develop-eggs directory is initially empty:
>>> ls(sample_buildout, 'develop-eggs') >>> ls(sample_buildout, 'develop-eggs')
>>> ls(sample_buildout, 'parts')
The develop-eggs directory holds egg links for software being The develop-eggs directory holds egg links for software being
developed in the buildout. We separate develop-eggs and other eggs to developed in the buildout. We separate develop-eggs and other eggs to
...@@ -69,6 +68,12 @@ directory in their home that all non-develop eggs are stored in. This ...@@ -69,6 +68,12 @@ directory in their home that all non-develop eggs are stored in. This
allows larger buildouts to be set up much more quickly and saves disk allows larger buildouts to be set up much more quickly and saves disk
space. space.
The parts directory just contains some helpers for the buildout script
itself.
>>> ls(sample_buildout, 'parts')
d buildout
The parts directory provides an area where recipes can install The parts directory provides an area where recipes can install
part data. For example, if we built a custom Python, we would part data. For example, if we built a custom Python, we would
install it in the part directory. Part data is stored in a install it in the part directory. Part data is stored in a
...@@ -576,7 +581,7 @@ When we rerun the buildout: ...@@ -576,7 +581,7 @@ When we rerun the buildout:
.. Wait for the file to really disappear. My linux is weird. .. Wait for the file to really disappear. My linux is weird.
>>> wait_until("foo goes away", lambda : not os.path.exists('foo'), >>> wait_until("foo goes away", lambda : not os.path.exists('foo'),
... timeout=100) ... timeout=200)
we get the same error, but we don't get the directory left behind: we get the same error, but we don't get the directory left behind:
...@@ -724,6 +729,10 @@ COMMAND_LINE_VALUE). ...@@ -724,6 +729,10 @@ COMMAND_LINE_VALUE).
================== ==================
<BLANKLINE> <BLANKLINE>
[buildout] [buildout]
allow-hosts= *
DEFAULT_VALUE
allow-picked-versions= true
DEFAULT_VALUE
bin-directory= bin bin-directory= bin
DEFAULT_VALUE DEFAULT_VALUE
develop= recipes develop= recipes
...@@ -736,18 +745,36 @@ COMMAND_LINE_VALUE). ...@@ -736,18 +745,36 @@ COMMAND_LINE_VALUE).
DEFAULT_VALUE DEFAULT_VALUE
executable= ... executable= ...
DEFAULT_VALUE DEFAULT_VALUE
find-links=
DEFAULT_VALUE
install-from-cache= false
DEFAULT_VALUE
installed= .installed.cfg installed= .installed.cfg
DEFAULT_VALUE DEFAULT_VALUE
log-format= log-format=
DEFAULT_VALUE DEFAULT_VALUE
log-level= INFO log-level= INFO
DEFAULT_VALUE DEFAULT_VALUE
newest= true
DEFAULT_VALUE
offline= false
DEFAULT_VALUE
parts= data-dir parts= data-dir
/sample-buildout/buildout.cfg /sample-buildout/buildout.cfg
parts-directory= parts parts-directory= parts
DEFAULT_VALUE DEFAULT_VALUE
prefer-final= false
DEFAULT_VALUE
python= buildout python= buildout
DEFAULT_VALUE DEFAULT_VALUE
relative-paths= false
DEFAULT_VALUE
socket-timeout=
DEFAULT_VALUE
unzip= false
DEFAULT_VALUE
use-dependency-links= true
DEFAULT_VALUE
<BLANKLINE> <BLANKLINE>
[data-dir] [data-dir]
path= foo bins path= foo bins
...@@ -2194,17 +2221,21 @@ database is shown. ...@@ -2194,17 +2221,21 @@ database is shown.
>>> print system(buildout+' -vv'), # doctest: +NORMALIZE_WHITESPACE >>> print system(buildout+' -vv'), # doctest: +NORMALIZE_WHITESPACE
Installing 'zc.buildout', 'setuptools'. Installing 'zc.buildout', 'setuptools'.
We have a develop egg: zc.buildout 1.0.0. We have a develop egg: zc.buildout X.X.
We have the best distribution that satisfies 'setuptools'. We have the best distribution that satisfies 'setuptools'.
Picked: setuptools = 0.6 Picked: setuptools = V.V
<BLANKLINE> <BLANKLINE>
Configuration data: Configuration data:
[buildout] [buildout]
allow-hosts = *
allow-picked-versions = true
bin-directory = /sample-buildout/bin bin-directory = /sample-buildout/bin
develop-eggs-directory = /sample-buildout/develop-eggs develop-eggs-directory = /sample-buildout/develop-eggs
directory = /sample-buildout directory = /sample-buildout
eggs-directory = /sample-buildout/eggs eggs-directory = /sample-buildout/eggs
executable = /usr/local/bin/python2.3 executable = python
find-links =
install-from-cache = false
installed = /sample-buildout/.installed.cfg installed = /sample-buildout/.installed.cfg
log-format = log-format =
log-level = INFO log-level = INFO
...@@ -2212,7 +2243,12 @@ database is shown. ...@@ -2212,7 +2243,12 @@ database is shown.
offline = false offline = false
parts = parts =
parts-directory = /sample-buildout/parts parts-directory = /sample-buildout/parts
prefer-final = false
python = buildout python = buildout
relative-paths = false
socket-timeout =
unzip = false
use-dependency-links = true
verbosity = 20 verbosity = 20
<BLANKLINE> <BLANKLINE>
...@@ -2273,6 +2309,33 @@ python ...@@ -2273,6 +2309,33 @@ python
Python executable. By default, the buildout section defines the Python executable. By default, the buildout section defines the
default Python as the Python used to run the buildout. default Python as the Python used to run the buildout.
relative-paths
The paths generated by zc.buildout are absolute by default, and this
option is ``false``. However, if you set this value to be ``true``,
bin/buildout will be generated with code that makes the paths relative.
Some recipes, such as zc.recipe.egg and z3c.recipe.scripts, honor this
value as well.
unzip
By default, zc.buildout doesn't unzip zip-safe eggs ("unzip = false").
This follows the policy followed by setuptools itself. Experience shows
this policy to to be inconvenient. Zipped eggs make debugging more
difficult and often import more slowly. You can include an unzip option in
the buildout section to change the default unzipping policy ("unzip =
true").
use-dependency-links
By default buildout will obey the setuptools dependency_links metadata
when it looks for dependencies. This behavior can be controlled with
the use-dependency-links buildout option::
[buildout]
...
use-dependency-links = false
The option defaults to true. If you set it to false, then dependency
links are only looked for in the locations specified by find-links.
verbosity verbosity
A log-level adjustment. Typically, this is set via the -q and -v A log-level adjustment. Typically, this is set via the -q and -v
command-line options. command-line options.
...@@ -2319,8 +2382,56 @@ buildout or setuptools egg could be installed in the develop-eggs ...@@ -2319,8 +2382,56 @@ buildout or setuptools egg could be installed in the develop-eggs
directory if the original buildout had develop eggs for either directory if the original buildout had develop eggs for either
buildout or setuptools.) buildout or setuptools.)
Note that the buildout script was installed but not run. To run If relative-paths is ``true``, the buildout script uses relative paths.
the buildout, we'd have to run the installed buildout script.
>>> write(sample_bootstrapped, 'setup.cfg',
... '''
... [buildout]
... relative-paths = true
... parts =
... ''')
>>> print system(buildout
... +' -c'+os.path.join(sample_bootstrapped, 'setup.cfg')
... +' bootstrap'),
Generated script '/sample-bootstrapped/bin/buildout'.
>>> buildout_script = join(sample_bootstrapped, 'bin', 'buildout')
>>> import sys
>>> if sys.platform.startswith('win'):
... buildout_script += '-script.py'
>>> print open(buildout_script).read() # doctest: +ELLIPSIS
#!... -S
<BLANKLINE>
import os
<BLANKLINE>
join = os.path.join
base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
base = os.path.dirname(base)
<BLANKLINE>
import sys
sys.path[0:0] = [
join(base, 'parts/buildout'),
]
<BLANKLINE>
<BLANKLINE>
import os
path = sys.path[0]
if os.environ.get('PYTHONPATH'):
path = os.pathsep.join([path, os.environ['PYTHONPATH']])
os.environ['PYTHONPATH'] = path
import site # imports custom buildout-generated site.py
<BLANKLINE>
import zc.buildout.buildout
<BLANKLINE>
if __name__ == '__main__':
zc.buildout.buildout.main()
<BLANKLINE>
Note that, in the above two examples, the buildout script was installed
but not run. To run the buildout, we'd have to run the installed
buildout script.
If we have an existing buildout that already has a buildout.cfg, we'll If we have an existing buildout that already has a buildout.cfg, we'll
normally use the bootstrap command instead of init. It will complain normally use the bootstrap command instead of init. It will complain
......
This diff is collapsed.
This diff is collapsed.
...@@ -28,6 +28,7 @@ import socket ...@@ -28,6 +28,7 @@ import socket
import subprocess import subprocess
import sys import sys
import tempfile import tempfile
import textwrap
import threading import threading
import time import time
import urllib2 import urllib2
...@@ -109,6 +110,16 @@ def system(command, input=''): ...@@ -109,6 +110,16 @@ def system(command, input=''):
e.close() e.close()
return result return result
def call_py(interpreter, cmd, flags=None):
if sys.platform == 'win32':
args = ['"%s"' % arg for arg in (interpreter, flags, cmd) if arg]
args.insert(-1, '"-c"')
return system('"%s"' % ' '.join(args))
else:
cmd = repr(cmd)
return system(
' '.join(arg for arg in (interpreter, flags, '-c', cmd) if arg))
def get(url): def get(url):
return urllib2.urlopen(url).read() return urllib2.urlopen(url).read()
...@@ -120,7 +131,11 @@ def _runsetup(setup, executable, *args): ...@@ -120,7 +131,11 @@ def _runsetup(setup, executable, *args):
args = [zc.buildout.easy_install._safe_arg(arg) args = [zc.buildout.easy_install._safe_arg(arg)
for arg in args] for arg in args]
args.insert(0, '-q') args.insert(0, '-q')
args.append(dict(os.environ, PYTHONPATH=setuptools_location)) env = dict(os.environ)
if executable == sys.executable:
env['PYTHONPATH'] = setuptools_location
# else pass an executable that has setuptools! See testselectingpython.py.
args.append(env)
here = os.getcwd() here = os.getcwd()
try: try:
...@@ -139,6 +154,11 @@ def sdist(setup, dest): ...@@ -139,6 +154,11 @@ def sdist(setup, dest):
def bdist_egg(setup, executable, dest): def bdist_egg(setup, executable, dest):
_runsetup(setup, executable, 'bdist_egg', '-d', dest) _runsetup(setup, executable, 'bdist_egg', '-d', dest)
def sys_install(setup, dest):
_runsetup(setup, sys.executable, 'install', '--install-purelib', dest,
'--record', os.path.join(dest, '__added_files__'),
'--single-version-externally-managed')
def find_python(version): def find_python(version):
e = os.environ.get('PYTHON%s' % version) e = os.environ.get('PYTHON%s' % version)
if e is not None: if e is not None:
...@@ -206,14 +226,64 @@ def wait_until(label, func, *args, **kw): ...@@ -206,14 +226,64 @@ def wait_until(label, func, *args, **kw):
time.sleep(0.01) time.sleep(0.01)
raise ValueError('Timed out waiting for: '+label) raise ValueError('Timed out waiting for: '+label)
def get_installer_values():
"""Get the current values for the easy_install module.
This is necessary because instantiating a Buildout will force the
Buildout's values on the installer.
Returns a dict of names-values suitable for set_installer_values."""
names = ('default_versions', 'download_cache', 'install_from_cache',
'prefer_final', 'include_site_packages',
'allowed_eggs_from_site_packages', 'use_dependency_links',
'allow_picked_versions', 'always_unzip'
)
values = {}
for name in names:
values[name] = getattr(zc.buildout.easy_install, name)()
return values
def set_installer_values(values):
"""Set the given values on the installer."""
for name, value in values.items():
getattr(zc.buildout.easy_install, name)(value)
def make_buildout(executable=None):
"""Make a buildout that uses this version of zc.buildout."""
# Create a basic buildout.cfg to avoid a warning from buildout.
open('buildout.cfg', 'w').write(
"[buildout]\nparts =\n"
)
# Get state of installer defaults so we can reinstate them (instantiating
# a Buildout will force the Buildout's defaults on the installer).
installer_values = get_installer_values()
# Use the buildout bootstrap command to create a buildout
config = [
('buildout', 'log-level', 'WARNING'),
# trick bootstrap into putting the buildout develop egg
# in the eggs dir.
('buildout', 'develop-eggs-directory', 'eggs'),
]
if executable is not None:
config.append(('buildout', 'executable', executable))
zc.buildout.buildout.Buildout(
'buildout.cfg', config,
user_defaults=False,
).bootstrap([])
# Create the develop-eggs dir, which didn't get created the usual
# way due to the trick above:
os.mkdir('develop-eggs')
# Reinstate the default values of the installer.
set_installer_values(installer_values)
def buildoutSetUp(test): def buildoutSetUp(test):
test.globs['__tear_downs'] = __tear_downs = [] test.globs['__tear_downs'] = __tear_downs = []
test.globs['register_teardown'] = register_teardown = __tear_downs.append test.globs['register_teardown'] = register_teardown = __tear_downs.append
prefer_final = zc.buildout.easy_install.prefer_final() installer_values = get_installer_values()
register_teardown( register_teardown(
lambda: zc.buildout.easy_install.prefer_final(prefer_final) lambda: set_installer_values(installer_values)
) )
here = os.getcwd() here = os.getcwd()
...@@ -259,27 +329,7 @@ def buildoutSetUp(test): ...@@ -259,27 +329,7 @@ def buildoutSetUp(test):
sample = tmpdir('sample-buildout') sample = tmpdir('sample-buildout')
os.chdir(sample) os.chdir(sample)
make_buildout()
# Create a basic buildout.cfg to avoid a warning from buildout:
open('buildout.cfg', 'w').write(
"[buildout]\nparts =\n"
)
# Use the buildout bootstrap command to create a buildout
zc.buildout.buildout.Buildout(
'buildout.cfg',
[('buildout', 'log-level', 'WARNING'),
# trick bootstrap into putting the buildout develop egg
# in the eggs dir.
('buildout', 'develop-eggs-directory', 'eggs'),
]
).bootstrap([])
# Create the develop-eggs dir, which didn't get created the usual
# way due to the trick above:
os.mkdir('develop-eggs')
def start_server(path): def start_server(path):
port, thread = _start_server(path, name=path) port, thread = _start_server(path, name=path)
...@@ -287,6 +337,50 @@ def buildoutSetUp(test): ...@@ -287,6 +337,50 @@ def buildoutSetUp(test):
register_teardown(lambda: stop_server(url, thread)) register_teardown(lambda: stop_server(url, thread))
return url return url
def make_py(initialization=''):
"""Returns paths to new executable and to its site-packages.
"""
buildout = tmpdir('executable_buildout')
site_packages_dir = os.path.join(buildout, 'site-packages')
mkdir(site_packages_dir)
old_wd = os.getcwd()
os.chdir(buildout)
make_buildout()
# Normally we don't process .pth files in extra-paths. We want to
# in this case so that we can test with setuptools system installs
# (--single-version-externally-managed), which use .pth files.
initialization = (
('import sys\n'
'import site\n'
'known_paths = set(sys.path)\n'
'site_packages_dir = %r\n'
'site.addsitedir(site_packages_dir, known_paths)\n'
) % (site_packages_dir,)) + initialization
initialization = '\n'.join(
' ' + line for line in initialization.split('\n'))
install_develop(
'zc.recipe.egg', os.path.join(buildout, 'develop-eggs'))
install_develop(
'z3c.recipe.scripts', os.path.join(buildout, 'develop-eggs'))
write('buildout.cfg', textwrap.dedent('''\
[buildout]
parts = py
[py]
recipe = z3c.recipe.scripts
interpreter = py
initialization =
%(initialization)s
extra-paths = %(site-packages)s
eggs = setuptools
''') % {
'initialization': initialization,
'site-packages': site_packages_dir})
system(os.path.join(buildout, 'bin', 'buildout'))
os.chdir(old_wd)
return (
os.path.join(buildout, 'bin', 'py'), site_packages_dir)
test.globs.update(dict( test.globs.update(dict(
sample_buildout = sample, sample_buildout = sample,
ls = ls, ls = ls,
...@@ -297,6 +391,7 @@ def buildoutSetUp(test): ...@@ -297,6 +391,7 @@ def buildoutSetUp(test):
tmpdir = tmpdir, tmpdir = tmpdir,
write = write, write = write,
system = system, system = system,
call_py = call_py,
get = get, get = get,
cd = (lambda *path: os.chdir(os.path.join(*path))), cd = (lambda *path: os.chdir(os.path.join(*path))),
join = os.path.join, join = os.path.join,
...@@ -305,10 +400,9 @@ def buildoutSetUp(test): ...@@ -305,10 +400,9 @@ def buildoutSetUp(test):
start_server = start_server, start_server = start_server,
buildout = os.path.join(sample, 'bin', 'buildout'), buildout = os.path.join(sample, 'bin', 'buildout'),
wait_until = wait_until, wait_until = wait_until,
make_py = make_py
)) ))
zc.buildout.easy_install.prefer_final(prefer_final)
def buildoutTearDown(test): def buildoutTearDown(test):
for f in test.globs['__tear_downs']: for f in test.globs['__tear_downs']:
f() f()
......
This diff is collapsed.
...@@ -11,7 +11,7 @@ ...@@ -11,7 +11,7 @@
# FOR A PARTICULAR PURPOSE. # FOR A PARTICULAR PURPOSE.
# #
############################################################################## ##############################################################################
import os, re, sys, unittest import os, re, subprocess, sys, textwrap, unittest
from zope.testing import doctest, renormalizing from zope.testing import doctest, renormalizing
import zc.buildout.tests import zc.buildout.tests
import zc.buildout.testing import zc.buildout.testing
...@@ -42,6 +42,33 @@ We can specify a specific Python executable. ...@@ -42,6 +42,33 @@ We can specify a specific Python executable.
def multi_python(test): def multi_python(test):
other_executable = zc.buildout.testing.find_python(other_version) other_executable = zc.buildout.testing.find_python(other_version)
command = textwrap.dedent('''\
try:
import setuptools
except ImportError:
import sys
sys.exit(1)
''')
if subprocess.call([other_executable, '-c', command],
env=os.environ):
# the other executable does not have setuptools. Get setuptools.
# We will do this using the same tools we are testing, for better or
# worse. Alternatively, we could try using bootstrap.
executable_dir = test.globs['tmpdir']('executable_dir')
executable_parts = os.path.join(executable_dir, 'parts')
test.globs['mkdir'](executable_parts)
ws = zc.buildout.easy_install.install(
['setuptools'], executable_dir,
index='http://www.python.org/pypi/',
always_unzip=True, executable=other_executable)
zc.buildout.easy_install.sitepackage_safe_scripts(
executable_dir, ws, other_executable, executable_parts,
reqs=['setuptools'], interpreter='py')
original_executable = other_executable
other_executable = os.path.join(executable_dir, 'py')
assert not subprocess.call(
[other_executable, '-c', command], env=os.environ), (
'test set up failed')
sample_eggs = test.globs['tmpdir']('sample_eggs') sample_eggs = test.globs['tmpdir']('sample_eggs')
os.mkdir(os.path.join(sample_eggs, 'index')) os.mkdir(os.path.join(sample_eggs, 'index'))
test.globs['sample_eggs'] = sample_eggs test.globs['sample_eggs'] = sample_eggs
......
...@@ -78,25 +78,30 @@ new versions found in new releases: ...@@ -78,25 +78,30 @@ new versions found in new releases:
zc.buildout 99.99 zc.buildout 99.99
setuptools 99.99 setuptools 99.99
Our buildout script has been updated to use the new eggs: Our buildout script's site.py has been updated to use the new eggs:
>>> cat(sample_buildout, 'bin', 'buildout') >>> cat(sample_buildout, 'parts', 'buildout', 'site.py')
#!/usr/local/bin/python2.4 ... # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS
"...
def addsitepackages(known_paths):
"""Add site packages, as determined by zc.buildout.
<BLANKLINE> <BLANKLINE>
import sys See original_addsitepackages, below, for the original version."""
sys.path[0:0] = [ buildout_paths = [
'/sample-buildout/eggs/zc.buildout-99.99-py2.4.egg', '/sample-buildout/eggs/zc.buildout-99.99-pyN.N.egg',
'/sample-buildout/eggs/setuptools-99.99-py2.4.egg', '/sample-buildout/eggs/setuptools-99.99-pyN.N.egg'
] ]
<BLANKLINE> for path in buildout_paths:
import zc.buildout.buildout sitedir, sitedircase = makepath(path)
<BLANKLINE> if not sitedircase in known_paths and os.path.exists(sitedir):
if __name__ == '__main__': sys.path.append(sitedir)
zc.buildout.buildout.main() known_paths.add(sitedircase)
return known_paths
...
Now, let's recreate the sample buildout. If we specify constraints on Now, let's recreate the sample buildout. If we specify constraints on
the versions of zc.buildout and setuptools to use, running the the versions of zc.buildout and setuptools (or distribute) to use,
buildout will install earlier versions of these packages: running the buildout will install earlier versions of these packages:
>>> write(sample_buildout, 'buildout.cfg', >>> write(sample_buildout, 'buildout.cfg',
... """ ... """
...@@ -107,6 +112,7 @@ buildout will install earlier versions of these packages: ...@@ -107,6 +112,7 @@ buildout will install earlier versions of these packages:
... develop = showversions ... develop = showversions
... zc.buildout-version = < 99 ... zc.buildout-version = < 99
... setuptools-version = < 99 ... setuptools-version = < 99
... distribute-version = < 99
... ...
... [show-versions] ... [show-versions]
... recipe = showversions ... recipe = showversions
...@@ -119,7 +125,6 @@ Now we can see that we actually "upgrade" to an earlier version. ...@@ -119,7 +125,6 @@ Now we can see that we actually "upgrade" to an earlier version.
zc.buildout version 1.0.0, zc.buildout version 1.0.0,
setuptools version 0.6; setuptools version 0.6;
restarting. restarting.
Generated script '/sample-buildout/bin/buildout'.
Develop: '/sample-buildout/showversions' Develop: '/sample-buildout/showversions'
Updating show-versions. Updating show-versions.
zc.buildout 1.0.0 zc.buildout 1.0.0
......
Change History
**************
1.0.0
=====
Initial public version.
********************************
Buildout Script Recipe
********************************
.. contents::
The script recipe installs eggs into a buildout eggs directory, exactly
like zc.recipe.egg, and then generates scripts in a buildout bin
directory with egg paths baked into them.
##############################################################################
#
# Copyright (c) 2007 Zope Corporation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
##############################################################################
"""Setup for z3c.recipe.scripts package
$Id: setup.py 106736 2009-12-18 02:33:08Z gary $
"""
version = '1.0.0dev'
import os
from setuptools import setup, find_packages
def read(*rnames):
return open(os.path.join(os.path.dirname(__file__), *rnames)).read()
name = "z3c.recipe.scripts"
setup(
name = name,
version = version,
author = "Gary Poster",
author_email = "gary.poster@canonical.com",
description = "Recipe for installing Python scripts",
long_description = (
read('README.txt')
+ '\n' +
read('CHANGES.txt')
+ '\n' +
'Detailed Documentation\n'
'**********************\n'
+ '\n' +
read('src', 'z3c', 'recipe', 'scripts', 'README.txt')
+ '\n' +
'Download\n'
'*********\n'
),
keywords = "development build",
classifiers = [
'Development Status :: 5 - Production/Stable',
'Framework :: Buildout',
'Intended Audience :: Developers',
'License :: OSI Approved :: Zope Public License',
'Topic :: Software Development :: Build Tools',
'Topic :: Software Development :: Libraries :: Python Modules',
],
url='http://cheeseshop.python.org/pypi/z3c.recipe.scripts',
license = "ZPL 2.1",
packages = find_packages('src'),
package_dir = {'':'src'},
namespace_packages = ['z3c', 'z3c.recipe'],
install_requires = [
'zc.buildout >=1.5.0dev',
'zc.recipe.egg >=1.2.3dev',
'setuptools'],
tests_require = ['zope.testing'],
test_suite = name+'.tests.test_suite',
entry_points = {'zc.buildout': ['default = %s:Scripts' % name,
'script = %s:Scripts' % name,
'scripts = %s:Scripts' % name,
'interpreter = %s:Interpreter' % name,
]
},
include_package_data = True,
zip_safe=False,
)
__import__('pkg_resources').declare_namespace(__name__)
__import__('pkg_resources').declare_namespace(__name__)
This diff is collapsed.
from z3c.recipe.scripts.scripts import Scripts, Interpreter
##############################################################################
#
# Copyright (c) 2009-2010 Zope Corporation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
##############################################################################
"""Install scripts from eggs.
"""
import os
import zc.buildout
import zc.buildout.easy_install
from zc.recipe.egg.egg import ScriptBase
class Base(ScriptBase):
def __init__(self, buildout, name, options):
if 'extends' in options:
options.update(buildout[options['extends']])
super(Base, self).__init__(buildout, name, options)
self.default_eggs = '' # Disables feature from zc.recipe.egg.
b_options = buildout['buildout']
options['parts-directory'] = os.path.join(
b_options['parts-directory'], self.name)
value = options.setdefault(
'allowed-eggs-from-site-packages',
'*')
self.allowed_eggs = tuple(name.strip() for name in value.split('\n'))
value = options.setdefault(
'include-site-packages',
b_options.get('include-site-packages', 'false'))
if value not in ('true', 'false'):
raise zc.buildout.UserError(
"Invalid value for include-site-packages option: %s" %
(value,))
self.include_site_packages = (value == 'true')
value = options.setdefault(
'exec-sitecustomize',
b_options.get('exec-sitecustomize', 'false'))
if value not in ('true', 'false'):
raise zc.buildout.UserError(
"Invalid value for exec-sitecustomize option: %s" %
(value,))
self.exec_sitecustomize = (value == 'true')
class Interpreter(Base):
def __init__(self, buildout, name, options):
super(Interpreter, self).__init__(buildout, name, options)
options.setdefault('name', name)
def install(self):
reqs, ws = self.working_set()
options = self.options
generated = []
if not os.path.exists(options['parts-directory']):
os.mkdir(options['parts-directory'])
generated.append(options['parts-directory'])
generated.extend(zc.buildout.easy_install.sitepackage_safe_scripts(
options['bin-directory'], ws, options['executable'],
options['parts-directory'],
interpreter=options['name'],
extra_paths=self.extra_paths,
initialization=options.get('initialization', ''),
include_site_packages=self.include_site_packages,
exec_sitecustomize=self.exec_sitecustomize,
relative_paths=self._relative_paths,
))
return generated
update = install
class Scripts(Base):
def _install(self, reqs, ws, scripts):
options = self.options
generated = []
if not os.path.exists(options['parts-directory']):
os.mkdir(options['parts-directory'])
generated.append(options['parts-directory'])
generated.extend(zc.buildout.easy_install.sitepackage_safe_scripts(
options['bin-directory'], ws, options['executable'],
options['parts-directory'], reqs=reqs, scripts=scripts,
interpreter=options.get('interpreter'),
extra_paths=self.extra_paths,
initialization=options.get('initialization', ''),
include_site_packages=self.include_site_packages,
exec_sitecustomize=self.exec_sitecustomize,
relative_paths=self._relative_paths,
script_arguments=options.get('arguments', ''),
script_initialization=options.get('script-initialization', '')
))
return generated
This diff is collapsed.
...@@ -16,7 +16,7 @@ ...@@ -16,7 +16,7 @@
$Id$ $Id$
""" """
version = '0' version = '1.2.3dev'
import os import os
from setuptools import setup, find_packages from setuptools import setup, find_packages
...@@ -66,7 +66,7 @@ setup( ...@@ -66,7 +66,7 @@ setup(
package_dir = {'':'src'}, package_dir = {'':'src'},
namespace_packages = ['zc', 'zc.recipe'], namespace_packages = ['zc', 'zc.recipe'],
install_requires = [ install_requires = [
'zc.buildout >=1.2.0', 'zc.buildout >=1.5.0dev',
'setuptools'], 'setuptools'],
tests_require = ['zope.testing'], tests_require = ['zope.testing'],
test_suite = name+'.tests.test_suite', test_suite = name+'.tests.test_suite',
......
...@@ -154,6 +154,8 @@ dependent-scripts ...@@ -154,6 +154,8 @@ dependent-scripts
interpreter interpreter
The name of a script to generate that allows access to a Python The name of a script to generate that allows access to a Python
interpreter that has the path set based on the eggs installed. interpreter that has the path set based on the eggs installed.
(See the ``z3c.recipe.scripts`` recipe for a more full-featured
interpreter.)
extra-paths extra-paths
Extra paths to include in a generated script. Extra paths to include in a generated script.
...@@ -577,7 +579,7 @@ declare entry points using the entry-points option: ...@@ -577,7 +579,7 @@ declare entry points using the entry-points option:
- demo - demo
- other - other
>>> cat(sample_buildout, 'bin', 'other') >>> cat(sample_buildout, 'bin', 'other') # doctest: +NORMALIZE_WHITESPACE
#!/usr/local/bin/python2.4 #!/usr/local/bin/python2.4
<BLANKLINE> <BLANKLINE>
import sys import sys
...@@ -640,3 +642,4 @@ be made to contact an index server: ...@@ -640,3 +642,4 @@ be made to contact an index server:
Uninstalling bigdemo. Uninstalling bigdemo.
Installing demo. Installing demo.
Generated script '/sample-buildout/bin/foo'. Generated script '/sample-buildout/bin/foo'.
...@@ -117,6 +117,7 @@ computed by the egg recipe by looking at .installed.cfg: ...@@ -117,6 +117,7 @@ computed by the egg recipe by looking at .installed.cfg:
extras = other extras = other
find-links = http://localhost:27071/ find-links = http://localhost:27071/
index = http://localhost:27071/index index = http://localhost:27071/index
python = buildout
recipe = sample recipe = sample
If we use the extra-paths option: If we use the extra-paths option:
......
...@@ -150,6 +150,7 @@ eggs directory can be shared across multiple buildouts. ...@@ -150,6 +150,7 @@ eggs directory can be shared across multiple buildouts.
>>> ls(sample_buildout, 'develop-eggs') >>> ls(sample_buildout, 'develop-eggs')
d extdemo-1.4-py2.4-unix-i686.egg d extdemo-1.4-py2.4-unix-i686.egg
- z3c.recipe.scripts.egg-link
- zc.recipe.egg.egg-link - zc.recipe.egg.egg-link
Note that no scripts or dependencies are installed. To install Note that no scripts or dependencies are installed. To install
...@@ -231,6 +232,7 @@ We won't get an update. ...@@ -231,6 +232,7 @@ We won't get an update.
>>> ls(sample_buildout, 'develop-eggs') >>> ls(sample_buildout, 'develop-eggs')
- demo.egg-link - demo.egg-link
d extdemo-1.4-py2.4-unix-i686.egg d extdemo-1.4-py2.4-unix-i686.egg
- z3c.recipe.scripts.egg-link
- zc.recipe.egg.egg-link - zc.recipe.egg.egg-link
But if we run the buildout in the default on-line and newest modes, we But if we run the buildout in the default on-line and newest modes, we
...@@ -248,6 +250,7 @@ version is imported: ...@@ -248,6 +250,7 @@ version is imported:
- demo.egg-link - demo.egg-link
d extdemo-1.4-py2.4-linux-i686.egg d extdemo-1.4-py2.4-linux-i686.egg
d extdemo-1.5-py2.4-linux-i686.egg d extdemo-1.5-py2.4-linux-i686.egg
- z3c.recipe.scripts.egg-link
- zc.recipe.egg.egg-link - zc.recipe.egg.egg-link
Controlling the version used Controlling the version used
...@@ -287,6 +290,7 @@ We can specify a specific version using the egg option: ...@@ -287,6 +290,7 @@ We can specify a specific version using the egg option:
>>> ls(sample_buildout, 'develop-eggs') >>> ls(sample_buildout, 'develop-eggs')
- demo.egg-link - demo.egg-link
d extdemo-1.4-py2.4-linux-i686.egg d extdemo-1.4-py2.4-linux-i686.egg
- z3c.recipe.scripts.egg-link
- zc.recipe.egg.egg-link - zc.recipe.egg.egg-link
...@@ -553,6 +557,7 @@ Our develop-eggs now includes an egg link for extdemo: ...@@ -553,6 +557,7 @@ Our develop-eggs now includes an egg link for extdemo:
>>> ls('develop-eggs') >>> ls('develop-eggs')
- demo.egg-link - demo.egg-link
- extdemo.egg-link - extdemo.egg-link
- z3c.recipe.scripts.egg-link
- zc.recipe.egg.egg-link - zc.recipe.egg.egg-link
and the extdemo now has a built extension: and the extdemo now has a built extension:
......
...@@ -19,14 +19,17 @@ $Id$ ...@@ -19,14 +19,17 @@ $Id$
import logging, os, re, zipfile import logging, os, re, zipfile
import zc.buildout.easy_install import zc.buildout.easy_install
class Eggs(object): class Eggs(object):
include_site_packages = allowed_eggs = None
def __init__(self, buildout, name, options): def __init__(self, buildout, name, options):
self.buildout = buildout self.buildout = buildout
self.name = name self.name = self.default_eggs = name
self.options = options self.options = options
links = options.get('find-links', b_options = buildout['buildout']
buildout['buildout'].get('find-links')) links = options.get('find-links', b_options['find-links'])
if links: if links:
links = links.split() links = links.split()
options['find-links'] = '\n'.join(links) options['find-links'] = '\n'.join(links)
...@@ -34,25 +37,25 @@ class Eggs(object): ...@@ -34,25 +37,25 @@ class Eggs(object):
links = () links = ()
self.links = links self.links = links
index = options.get('index', buildout['buildout'].get('index')) index = options.get('index', b_options.get('index'))
if index is not None: if index is not None:
options['index'] = index options['index'] = index
self.index = index self.index = index
allow_hosts = buildout['buildout'].get('allow-hosts', '*') allow_hosts = b_options['allow-hosts']
allow_hosts = tuple([host.strip() for host in allow_hosts.split('\n') allow_hosts = tuple([host.strip() for host in allow_hosts.split('\n')
if host.strip()!='']) if host.strip()!=''])
self.allow_hosts = allow_hosts self.allow_hosts = allow_hosts
options['eggs-directory'] = buildout['buildout']['eggs-directory'] options['eggs-directory'] = b_options['eggs-directory']
options['_e'] = options['eggs-directory'] # backward compat. options['_e'] = options['eggs-directory'] # backward compat.
options['develop-eggs-directory' options['develop-eggs-directory'] = b_options['develop-eggs-directory']
] = buildout['buildout']['develop-eggs-directory']
options['_d'] = options['develop-eggs-directory'] # backward compat. options['_d'] = options['develop-eggs-directory'] # backward compat.
assert options.get('unzip') in ('true', 'false', None) # verify that this is None, 'true' or 'false'
get_bool(options, 'unzip')
python = options.get('python', buildout['buildout']['python']) python = options.setdefault('python', b_options['python'])
options['executable'] = buildout[python]['executable'] options['executable'] = buildout[python]['executable']
def working_set(self, extra=()): def working_set(self, extra=()):
...@@ -61,31 +64,36 @@ class Eggs(object): ...@@ -61,31 +64,36 @@ class Eggs(object):
This is intended for reuse by similar recipes. This is intended for reuse by similar recipes.
""" """
options = self.options options = self.options
b_options = self.buildout['buildout']
distributions = [ distributions = [
r.strip() r.strip()
for r in options.get('eggs', self.name).split('\n') for r in options.get('eggs', self.default_eggs).split('\n')
if r.strip()] if r.strip()]
orig_distributions = distributions[:] orig_distributions = distributions[:]
distributions.extend(extra) distributions.extend(extra)
if self.buildout['buildout'].get('offline') == 'true': if b_options.get('offline') == 'true':
ws = zc.buildout.easy_install.working_set( ws = zc.buildout.easy_install.working_set(
distributions, options['executable'], distributions, options['executable'],
[options['develop-eggs-directory'], options['eggs-directory']] [options['develop-eggs-directory'],
options['eggs-directory']],
include_site_packages=self.include_site_packages,
allowed_eggs_from_site_packages=self.allowed_eggs,
) )
else: else:
kw = {} kw = {}
if options.get('unzip'): if 'unzip' in options:
kw['always_unzip'] = get_bool(options, 'unzip') kw['always_unzip'] = get_bool(options, 'unzip')
ws = zc.buildout.easy_install.install( ws = zc.buildout.easy_install.install(
distributions, options['eggs-directory'], distributions, options['eggs-directory'],
links=self.links, links=self.links,
index=self.index, index=self.index,
executable=options['executable'], executable=options['executable'],
path=[options['develop-eggs-directory']], path=[options['develop-eggs-directory']],
newest=self.buildout['buildout'].get('newest') == 'true', newest=b_options.get('newest') == 'true',
include_site_packages=self.include_site_packages,
allowed_eggs_from_site_packages=self.allowed_eggs,
allow_hosts=self.allow_hosts, allow_hosts=self.allow_hosts,
**kw) **kw)
...@@ -97,16 +105,19 @@ class Eggs(object): ...@@ -97,16 +105,19 @@ class Eggs(object):
update = install update = install
class Scripts(Eggs):
class ScriptBase(Eggs):
def __init__(self, buildout, name, options): def __init__(self, buildout, name, options):
super(Scripts, self).__init__(buildout, name, options) super(ScriptBase, self).__init__(buildout, name, options)
b_options = buildout['buildout']
options['bin-directory'] = buildout['buildout']['bin-directory'] options['bin-directory'] = b_options['bin-directory']
options['_b'] = options['bin-directory'] # backward compat. options['_b'] = options['bin-directory'] # backward compat.
self.extra_paths = [ self.extra_paths = [
os.path.join(buildout['buildout']['directory'], p.strip()) os.path.join(b_options['directory'], p.strip())
for p in options.get('extra-paths', '').split('\n') for p in options.get('extra-paths', '').split('\n')
if p.strip() if p.strip()
] ]
...@@ -115,11 +126,9 @@ class Scripts(Eggs): ...@@ -115,11 +126,9 @@ class Scripts(Eggs):
relative_paths = options.get( relative_paths = options.get(
'relative-paths', 'relative-paths', b_options.get('relative-paths', 'false'))
buildout['buildout'].get('relative-paths', 'false')
)
if relative_paths == 'true': if relative_paths == 'true':
options['buildout-directory'] = buildout['buildout']['directory'] options['buildout-directory'] = b_options['directory']
self._relative_paths = options['buildout-directory'] self._relative_paths = options['buildout-directory']
else: else:
self._relative_paths = '' self._relative_paths = ''
...@@ -128,12 +137,13 @@ class Scripts(Eggs): ...@@ -128,12 +137,13 @@ class Scripts(Eggs):
parse_entry_point = re.compile( parse_entry_point = re.compile(
'([^=]+)=(\w+(?:[.]\w+)*):(\w+(?:[.]\w+)*)$' '([^=]+)=(\w+(?:[.]\w+)*):(\w+(?:[.]\w+)*)$'
).match ).match
def install(self): def install(self):
reqs, ws = self.working_set() reqs, ws = self.working_set()
options = self.options options = self.options
scripts = options.get('scripts') scripts = options.get('scripts')
if scripts or scripts is None: if scripts or scripts is None or options.get('interpreter'):
if scripts is not None: if scripts is not None:
scripts = scripts.split() scripts = scripts.split()
scripts = dict([ scripts = dict([
...@@ -157,22 +167,32 @@ class Scripts(Eggs): ...@@ -157,22 +167,32 @@ class Scripts(Eggs):
name = dist.project_name name = dist.project_name
if name != 'setuptools' and name not in reqs: if name != 'setuptools' and name not in reqs:
reqs.append(name) reqs.append(name)
return self._install(reqs, ws, scripts)
return zc.buildout.easy_install.scripts(
reqs, ws, options['executable'],
options['bin-directory'],
scripts=scripts,
extra_paths=self.extra_paths,
interpreter=options.get('interpreter'),
initialization=options.get('initialization', ''),
arguments=options.get('arguments', ''),
relative_paths=self._relative_paths,
)
return () return ()
update = install update = install
def _install(self, reqs, ws, scripts):
# Subclasses implement this.
raise NotImplementedError()
class Scripts(ScriptBase):
def _install(self, reqs, ws, scripts):
options = self.options
return zc.buildout.easy_install.scripts(
reqs, ws, options['executable'],
options['bin-directory'],
scripts=scripts,
extra_paths=self.extra_paths,
interpreter=options.get('interpreter'),
initialization=options.get('initialization', ''),
arguments=options.get('arguments', ''),
relative_paths=self._relative_paths
)
def get_bool(options, name, default=False): def get_bool(options, name, default=False):
value = options.get(name) value = options.get(name)
if not value: if not value:
......
...@@ -35,7 +35,7 @@ install the demo package using Python 2.4. ...@@ -35,7 +35,7 @@ install the demo package using Python 2.4.
... index = http://www.python.org/pypi/ ... index = http://www.python.org/pypi/
... ...
... [python2.4] ... [python2.4]
... executable = %(python23)s ... executable = %(python24)s
... ...
... [demo] ... [demo]
... recipe = zc.recipe.egg ... recipe = zc.recipe.egg
...@@ -43,7 +43,7 @@ install the demo package using Python 2.4. ...@@ -43,7 +43,7 @@ install the demo package using Python 2.4.
... find-links = %(server)s ... find-links = %(server)s
... python = python2.4 ... python = python2.4
... interpreter = py-demo ... interpreter = py-demo
... """ % dict(server=link_server, python23=other_executable)) ... """ % dict(server=link_server, python24=other_executable))
Now, if we run the buildout: Now, if we run the buildout:
......
...@@ -50,9 +50,12 @@ def test_suite(): ...@@ -50,9 +50,12 @@ def test_suite():
zc.buildout.tests.normalize_bang, zc.buildout.tests.normalize_bang,
(re.compile('zc.buildout(-\S+)?[.]egg(-link)?'), (re.compile('zc.buildout(-\S+)?[.]egg(-link)?'),
'zc.buildout.egg'), 'zc.buildout.egg'),
(re.compile('[-d] setuptools-[^-]+-'), 'setuptools-X-'), (re.compile('[-d] (setuptools|distribute)-[^-]+-'),
'setuptools-X-'),
(re.compile(r'eggs\\\\demo'), 'eggs/demo'), (re.compile(r'eggs\\\\demo'), 'eggs/demo'),
(re.compile(r'[a-zA-Z]:\\\\foo\\\\bar'), '/foo/bar'), (re.compile(r'[a-zA-Z]:\\\\foo\\\\bar'), '/foo/bar'),
# Distribute unzips eggs by default.
(re.compile('\- demoneeded'), 'd demoneeded'),
]) ])
), ),
doctest.DocFileSuite( doctest.DocFileSuite(
...@@ -64,7 +67,7 @@ def test_suite(): ...@@ -64,7 +67,7 @@ def test_suite():
(re.compile('__buildout_signature__ = ' (re.compile('__buildout_signature__ = '
'sample-\S+\s+' 'sample-\S+\s+'
'zc.recipe.egg-\S+\s+' 'zc.recipe.egg-\S+\s+'
'setuptools-\S+\s+' '(setuptools|distribute)-\S+\s+'
'zc.buildout-\S+\s*' 'zc.buildout-\S+\s*'
), ),
'__buildout_signature__ = sample- zc.recipe.egg-\n'), '__buildout_signature__ = sample- zc.recipe.egg-\n'),
...@@ -104,14 +107,17 @@ def test_suite(): ...@@ -104,14 +107,17 @@ def test_suite():
zc.buildout.testing.normalize_path, zc.buildout.testing.normalize_path,
zc.buildout.testing.normalize_endings, zc.buildout.testing.normalize_endings,
zc.buildout.testing.normalize_script, zc.buildout.testing.normalize_script,
(re.compile('Got setuptools \S+'), 'Got setuptools V'), (re.compile('Got (setuptools|distribute) \S+'),
(re.compile('([d-] )?setuptools-\S+-py'), 'Got setuptools V'),
(re.compile('([d-] )?(setuptools|distribute)-\S+-py'),
'setuptools-V-py'), 'setuptools-V-py'),
(re.compile('-py2[.][0-35-9][.]'), 'py2.5.'), (re.compile('-py2[.][0-35-9][.]'), 'py2.5.'),
(re.compile('zc.buildout-\S+[.]egg'), (re.compile('zc.buildout-\S+[.]egg'),
'zc.buildout.egg'), 'zc.buildout.egg'),
(re.compile('zc.buildout[.]egg-link'), (re.compile('zc.buildout[.]egg-link'),
'zc.buildout.egg'), 'zc.buildout.egg'),
# Distribute unzips eggs by default.
(re.compile('\- demoneeded'), 'd demoneeded'),
]), ]),
), ),
) )
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment