Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
S
slapos.buildout
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
6
Merge Requests
6
Analytics
Analytics
Repository
Value Stream
Wiki
Wiki
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Commits
Issue Boards
Open sidebar
nexedi
slapos.buildout
Commits
6107db1f
Commit
6107db1f
authored
Feb 10, 2010
by
Gary Poster
Browse files
Options
Browse Files
Download
Plain Diff
A -S branch
parents
337b7586
a542e58e
Changes
31
Hide whitespace changes
Inline
Side-by-side
Showing
31 changed files
with
562 additions
and
2691 deletions
+562
-2691
CHANGES.txt
CHANGES.txt
+29
-41
README.txt
README.txt
+2
-8
bootstrap/bootstrap.py
bootstrap/bootstrap.py
+109
-98
buildout.cfg
buildout.cfg
+2
-3
dev.py
dev.py
+1
-1
setup.py
setup.py
+4
-2
src/zc/buildout/bootstrap.txt
src/zc/buildout/bootstrap.txt
+108
-32
src/zc/buildout/buildout.py
src/zc/buildout/buildout.py
+2
-1
src/zc/buildout/distribute.txt
src/zc/buildout/distribute.txt
+23
-0
src/zc/buildout/easy_install.py
src/zc/buildout/easy_install.py
+95
-520
src/zc/buildout/easy_install.txt
src/zc/buildout/easy_install.txt
+37
-599
src/zc/buildout/testing.py
src/zc/buildout/testing.py
+24
-83
src/zc/buildout/tests.py
src/zc/buildout/tests.py
+2
-281
src/zc/buildout/testselectingpython.py
src/zc/buildout/testselectingpython.py
+1
-29
src/zc/buildout/update.txt
src/zc/buildout/update.txt
+4
-5
src/zc/buildout/upgrading_distribute.txt
src/zc/buildout/upgrading_distribute.txt
+56
-0
z3c.recipe.scripts_/CHANGES.txt
z3c.recipe.scripts_/CHANGES.txt
+0
-7
z3c.recipe.scripts_/README.txt
z3c.recipe.scripts_/README.txt
+0
-10
z3c.recipe.scripts_/setup.py
z3c.recipe.scripts_/setup.py
+0
-76
z3c.recipe.scripts_/src/z3c/__init__.py
z3c.recipe.scripts_/src/z3c/__init__.py
+0
-1
z3c.recipe.scripts_/src/z3c/recipe/__init__.py
z3c.recipe.scripts_/src/z3c/recipe/__init__.py
+0
-1
z3c.recipe.scripts_/src/z3c/recipe/scripts/README.txt
z3c.recipe.scripts_/src/z3c/recipe/scripts/README.txt
+0
-406
z3c.recipe.scripts_/src/z3c/recipe/scripts/__init__.py
z3c.recipe.scripts_/src/z3c/recipe/scripts/__init__.py
+0
-1
z3c.recipe.scripts_/src/z3c/recipe/scripts/scripts.py
z3c.recipe.scripts_/src/z3c/recipe/scripts/scripts.py
+0
-101
z3c.recipe.scripts_/src/z3c/recipe/scripts/tests.py
z3c.recipe.scripts_/src/z3c/recipe/scripts/tests.py
+0
-296
zc.recipe.egg_/src/zc/recipe/egg/README.txt
zc.recipe.egg_/src/zc/recipe/egg/README.txt
+12
-17
zc.recipe.egg_/src/zc/recipe/egg/api.txt
zc.recipe.egg_/src/zc/recipe/egg/api.txt
+3
-5
zc.recipe.egg_/src/zc/recipe/egg/custom.txt
zc.recipe.egg_/src/zc/recipe/egg/custom.txt
+9
-14
zc.recipe.egg_/src/zc/recipe/egg/egg.py
zc.recipe.egg_/src/zc/recipe/egg/egg.py
+27
-41
zc.recipe.egg_/src/zc/recipe/egg/selecting-python.txt
zc.recipe.egg_/src/zc/recipe/egg/selecting-python.txt
+8
-8
zc.recipe.egg_/src/zc/recipe/egg/tests.py
zc.recipe.egg_/src/zc/recipe/egg/tests.py
+4
-4
No files found.
CHANGES.txt
View file @
6107db1f
...
...
@@ -6,34 +6,6 @@ Change History
New Features:
- Buildout can be safely used with a system Python. Note that Python is
always used, by default, with -S: that is, site-packages are not
included. If you would like to have access to your site-packages, see
the next bullet point.
A limitation: in no cases are distributions in your site-packages used
to satisfy buildout dependencies. The site-packages can be used in
addition to the dependencies specified in your buildout, and buildout
dependencies can override code in your site-packages, but even if your
Python's site-packages has the same exact version as specified in your
buildout configuration, buildout will still use its own copy.
- Added new function, ``zc.buildout.easy_install.generate_scripts``, to
generate scripts and interpreter. It produces a full-featured
interpreter (all command-line options supported) and the ability to
safely let scripts include site packages. A safe and functional
site.py also is available, which will make it safer for use with a
system Python than the pre-existing buildout function
(``zc.buildout.easy_install.scripts``) and recipe (``zc.recipe.egg``).
End users should see z3c.recipe.scripts, which provides this
functionality within buildout configuration files.
The older function (``zc.buildout.easy_install.scripts``) and recipe
(``zc.recipe.egg``) still survive, because of their comparative attractive
simplicity, because they still work well when a non-system Python is used,
and because they can often be used well with a system Python.
- Improve bootstrap.
* New options let you specify where to find ez_setup.py and where to find
...
...
@@ -47,24 +19,40 @@ New Features:
Bugs fixed:
- Incrementing didn't work properly when extending multiple files.
https://bugs.launchpad.net/zc.buildout/+bug/421022
- The download API computed MD5 checksums of text files wrong on Windows.
- The handling and documenting of default buildout options was normalized.
This means, among other things, that ``bin/buildout -vv`` and
``bin/buildout annotate`` correctly list more of the options.
- Installing a namespace package using a Python that already has a package
in the same namespace (e.g., in the Python's site-packages) in some cases.
1.4.3 (2009-12-10)
==================
Bugs fixed:
- The error showed itself when at least two dependencies were in a shared
location like site-packages, and the first one met the "versions"
setting. The first dependency would be added, but subsequent
dependencies from the same location (e.g., site-packages) would use
the version of the package found in the shared location, ignoring the
version setting.
- Using pre-detected setuptools version for easy_installing tgz files. This
prevents a recursion error when easy_installing an upgraded "distribute"
tgz. Note that setuptools did not have this recursion problem solely
because it was packaged as an ``.egg``, which does not have to go through
the easy_install step.
1.4.2 (2009-11-01)
==================
New Feature:
- Added a --distribute option to the bootstrap script, in order
to use Distribute rather than Setuptools. By default, Setuptools
is used.
Bugs fixed:
- While checking for new versions of setuptools and buildout itself,
compare requirement locations instead of requirement objects.
- Incrementing didn't work properly when extending multiple files.
https://bugs.launchpad.net/zc.buildout/+bug/421022
- The download API computed MD5 checksums of text files wrong on Windows.
1.4.1 (2009-08-27)
==================
...
...
README.txt
View file @
6107db1f
...
...
@@ -37,11 +37,6 @@ Existing recipes include:
dependencies. It installs their console-script entry points with
the needed eggs included in their paths.
`z3c.recipe.scripts <http://pypi.python.org/pypi/z3c.recipe.scripts>`_
This scripts recipe builds interpreter scripts and entry point scripts
based on eggs. These scripts have more features and flexibility than the
ones offered by zc.recipe.egg.
`zc.recipe.testrunner <http://pypi.python.org/pypi/zc.recipe.testrunner>`_
The testrunner egg creates a test runner script for one or
more eggs.
...
...
@@ -131,7 +126,7 @@ If I need to run a previous version of zc.buildout, I use the
`--version` option of the bootstrap.py script::
$ python bootstrap.py --version 1.1.3
The `zc.buildout project <http://svn.zope.org/zc.buildout/trunk>`_
is a slightly more complex example of this type of buildout.
...
...
@@ -196,5 +191,4 @@ Please send questions and comments to the
`distutils SIG mailing list <mailto://distutils-sig@python.org>`_.
Report bugs using the `zc.buildout Launchpad Bug Tracker
<https://launchpad.net/products/zc.buildout/+bugs>`_.
<https://launchpad.net/zc.buildout/+bugs>`_.
bootstrap/bootstrap.py
View file @
6107db1f
...
...
@@ -20,105 +20,109 @@ use the -c option to specify an alternate configuration file.
$Id$
"""
import
os
,
re
,
shutil
,
sys
,
tempfile
,
textwrap
,
urllib
,
urllib2
# We have to manually parse our options rather than using one of the stdlib
# tools because we want to pass the ones we don't recognize along to
# zc.buildout.buildout.main.
configuration
=
{
'--ez_setup-source'
:
'http://peak.telecommunity.com/dist/ez_setup.py'
,
'--version'
:
''
,
'--download-base'
:
None
,
'--eggs'
:
None
}
helpstring
=
__doc__
+
textwrap
.
dedent
(
'''
This script recognizes the following options itself. The first option it
encounters that is not one of these will cause the script to stop parsing
options and pass the rest on to buildout. Therefore, if you want to use
any of the following options *and* buildout command-line options like
-c, first use the following options, and then use the buildout options.
Options:
--version=ZC_BUILDOUT_VERSION
Specify a version number of the zc.buildout to use
--ez_setup-source=URL_OR_FILE
Specify a URL or file location for the ez_setup file.
Defaults to
%(--ez_setup-source)s
--download-base=URL_OR_DIRECTORY
Specify a URL or directory for downloading setuptools and
zc.buildout. Defaults to PyPI.
--eggs=DIRECTORY
Specify a directory for storing eggs. Defaults to a temporary
directory that is deleted when the bootstrap script completes.
By using --ez_setup-source and --download-base to point to local resources,
you can keep this script from going over the network.
'''
%
configuration
)
match_equals
=
re
.
compile
(
r'(%s)=(.*)'
%
(
'|'
.
join
(
configuration
),)).
match
args
=
sys
.
argv
[
1
:]
if
args
==
[
'--help'
]:
print
helpstring
sys
.
exit
(
0
)
# If we end up using a temporary directory for storing our eggs, this will
# hold the path of that directory. On the other hand, if an explicit directory
# is specified in the argv, this will remain None.
tmpeggs
=
None
while
args
:
val
=
args
[
0
]
if
val
in
configuration
:
del
args
[
0
]
if
not
args
or
args
[
0
].
startswith
(
'-'
):
print
"ERROR: %s requires an argument."
print
helpstring
sys
.
exit
(
1
)
configuration
[
val
]
=
args
[
0
]
import
os
,
shutil
,
sys
,
tempfile
,
textwrap
,
urllib
,
urllib2
from
optparse
import
OptionParser
is_jython
=
sys
.
platform
.
startswith
(
'java'
)
setuptools_source
=
'http://peak.telecommunity.com/dist/ez_setup.py'
distribute_source
=
'http://python-distribute.org/distribute_setup.py'
# parsing arguments
def
normalize_to_url
(
option
,
opt_str
,
value
,
parser
):
if
value
:
if
'://'
not
in
value
:
# It doesn't smell like a URL.
value
=
'file://%s'
%
(
urllib
.
pathname2url
(
os
.
path
.
abspath
(
os
.
path
.
expanduser
(
value
))),)
if
opt_str
==
'--download-base'
and
not
value
.
endswith
(
'/'
):
# Download base needs a trailing slash to make the world happy.
value
+=
'/'
else
:
match
=
match_equals
(
val
)
if
match
and
match
.
group
(
1
)
in
configuration
:
configuration
[
match
.
group
(
1
)]
=
match
.
group
(
2
)
else
:
break
del
args
[
0
]
for
name
in
(
'--ez_setup-source'
,
'--download-base'
):
val
=
configuration
[
name
]
if
val
is
not
None
and
'://'
not
in
val
:
# We're being lazy.
configuration
[
name
]
=
'file://%s'
%
(
urllib
.
pathname2url
(
os
.
path
.
abspath
(
os
.
path
.
expanduser
(
val
))),)
if
(
configuration
[
'--download-base'
]
and
not
configuration
[
'--download-base'
].
endswith
(
'/'
)):
# Download base needs a trailing slash to make the world happy.
configuration
[
'--download-base'
]
+=
'/'
if
not
configuration
[
'--eggs'
]:
configuration
[
'--eggs'
]
=
tmpeggs
=
tempfile
.
mkdtemp
()
value
=
None
name
=
opt_str
[
2
:].
replace
(
'-'
,
'_'
)
setattr
(
parser
.
values
,
name
,
value
)
usage
=
'''
\
[DESIRED PYTHON FOR BUILDOUT] bootstrap.py [options]
Bootstraps a buildout-based project.
Simply run this script in a directory containing a buildout.cfg, using the
Python that you want bin/buildout to use.
Note that by using --setup-source and --download-base to point to
local resources, you can keep this script from going over the network.
'''
parser
=
OptionParser
(
usage
=
usage
)
parser
.
add_option
(
"-v"
,
"--version"
,
dest
=
"version"
,
help
=
"use a specific zc.buildout version"
)
parser
.
add_option
(
"-d"
,
"--distribute"
,
action
=
"store_true"
,
dest
=
"use_distribute"
,
default
=
False
,
help
=
"Use Distribute rather than Setuptools."
)
parser
.
add_option
(
"--setup-source"
,
action
=
"callback"
,
dest
=
"setup_source"
,
callback
=
normalize_to_url
,
nargs
=
1
,
type
=
"string"
,
help
=
(
"Specify a URL or file location for the setup file. "
"If you use Setuptools, this will default to "
+
setuptools_source
+
"; if you use Distribute, this "
"will default to "
+
distribute_source
+
"."
))
parser
.
add_option
(
"--download-base"
,
action
=
"callback"
,
dest
=
"download_base"
,
callback
=
normalize_to_url
,
nargs
=
1
,
type
=
"string"
,
help
=
(
"Specify a URL or directory for downloading "
"zc.buildout and either Setuptools or Distribute. "
"Defaults to PyPI."
))
parser
.
add_option
(
"--eggs"
,
help
=
(
"Specify a directory for storing eggs. Defaults to "
"a temporary directory that is deleted when the "
"bootstrap script completes."
))
parser
.
add_option
(
"-c"
,
None
,
action
=
"store"
,
dest
=
"config_file"
,
help
=
(
"Specify the path to the buildout configuration "
"file to be used."
))
options
,
args
=
parser
.
parse_args
()
# if -c was provided, we push it back into args for buildout' main function
if
options
.
config_file
is
not
None
:
args
+=
[
'-c'
,
options
.
config_file
]
if
options
.
eggs
:
eggs_dir
=
os
.
path
.
abspath
(
os
.
path
.
expanduser
(
options
.
eggs
))
else
:
configuration
[
'--eggs'
]
=
os
.
path
.
abspath
(
os
.
path
.
expanduser
(
configuration
[
'--eggs'
]))
eggs_dir
=
tempfile
.
mkdtemp
()
# The requirement is what we will pass to setuptools to specify zc.buildout.
requirement
=
'zc.buildout'
if
configuration
[
'--version'
]:
requirement
+=
'=='
+
configuration
[
'--version'
]
if
options
.
setup_source
is
None
:
if
options
.
use_distribute
:
options
.
setup_source
=
distribute_source
else
:
options
.
setup_source
=
setuptools_source
args
=
args
+
[
'bootstrap'
]
to_reload
=
False
try
:
import
setuptools
# A flag. Sometimes pkg_resources is installed alone.
import
pkg_resources
if
not
hasattr
(
pkg_resources
,
'_distribute'
):
to_reload
=
True
raise
ImportError
except
ImportError
:
ez_code
=
urllib2
.
urlopen
(
configuration
[
'--ez_setup-source'
]
).
read
().
replace
(
'
\
r
\
n
'
,
'
\
n
'
)
options
.
setup_source
).
read
().
replace
(
'
\
r
\
n
'
,
'
\
n
'
)
ez
=
{}
exec
ez_code
in
ez
setuptools_args
=
dict
(
to_dir
=
configuration
[
'--eggs'
],
download_delay
=
0
)
if
configuration
[
'--download-base'
]:
setuptools_args
[
'download_base'
]
=
configuration
[
'--download-base'
]
ez
[
'use_setuptools'
](
**
setuptools_args
)
import
pkg_resources
setup_args
=
dict
(
to_dir
=
eggs_dir
,
download_delay
=
0
)
if
options
.
download_base
:
setup_args
[
'download_base'
]
=
options
.
download_base
if
options
.
use_distribute
:
setup_args
[
'no_fake'
]
=
True
ez
[
'use_setuptools'
](
**
setup_args
)
if
to_reload
:
reload
(
pkg_resources
)
else
:
import
pkg_resources
# This does not (always?) update the default working set. We will
# do it.
for
path
in
sys
.
path
:
...
...
@@ -134,27 +138,35 @@ if sys.platform == 'win32':
else
:
def
quote
(
c
):
return
c
cmd
=
[
quote
(
sys
.
executable
),
'-c'
,
quote
(
'from setuptools.command.easy_install import main; main()'
),
'-mqNxd'
,
quote
(
configuration
[
'--eggs'
]
)]
quote
(
eggs_dir
)]
if
configuration
[
'--download-base'
]
:
cmd
.
extend
([
'-f'
,
quote
(
configuration
[
'--download-base'
]
)])
if
options
.
download_base
:
cmd
.
extend
([
'-f'
,
quote
(
options
.
download_base
)])
requirement
=
'zc.buildout'
if
options
.
version
:
requirement
=
'=='
.
join
((
requirement
,
options
.
version
))
cmd
.
append
(
requirement
)
if
options
.
use_distribute
:
setup_requirement
=
'distribute'
else
:
setup_requirement
=
'setuptools'
ws
=
pkg_resources
.
working_set
env
=
dict
(
os
.
environ
,
PYTHONPATH
=
ws
.
find
(
pkg_resources
.
Requirement
.
parse
(
'setuptools'
)).
location
)
PYTHONPATH
=
ws
.
find
(
pkg_resources
.
Requirement
.
parse
(
setup_requirement
)).
location
)
is_jython
=
sys
.
platform
.
startswith
(
'java'
)
if
is_jython
:
import
subprocess
exitcode
=
subprocess
.
Popen
(
cmd
,
env
=
env
).
wait
()
else
:
# Windows
need
s this, apparently; otherwise we would prefer subprocess
else
:
# Windows
prefer
s this, apparently; otherwise we would prefer subprocess
exitcode
=
os
.
spawnle
(
*
([
os
.
P_WAIT
,
sys
.
executable
]
+
cmd
+
[
env
]))
if
exitcode
!=
0
:
sys
.
stdout
.
flush
()
...
...
@@ -164,10 +176,9 @@ if exitcode != 0:
"were output by easy_install."
)
sys
.
exit
(
exitcode
)
ws
.
add_entry
(
configuration
[
'--eggs'
]
)
ws
.
add_entry
(
eggs_dir
)
ws
.
require
(
requirement
)
import
zc.buildout.buildout
args
.
append
(
'bootstrap'
)
zc
.
buildout
.
buildout
.
main
(
args
)
if
tmpeggs
is
not
None
:
shutil
.
rmtree
(
tmpeggs
)
if
not
options
.
eggs
:
# clean up temporary egg directory
shutil
.
rmtree
(
eggs_dir
)
buildout.cfg
View file @
6107db1f
[buildout]
develop = zc.recipe.egg_
z3c.recipe.scripts_
.
develop = zc.recipe.egg_ .
parts = test oltest py
[py]
...
...
@@ -13,7 +13,6 @@ recipe = zc.recipe.testrunner
eggs =
zc.buildout
zc.recipe.egg
z3c.recipe.scripts
# Tests that can be run wo a network
[oltest]
...
...
@@ -21,9 +20,9 @@ recipe = zc.recipe.testrunner
eggs =
zc.buildout
zc.recipe.egg
z3c.recipe.scripts
defaults =
[
'-t',
'!(bootstrap|selectingpython|selecting-python)',
]
dev.py
View file @
6107db1f
...
...
@@ -13,7 +13,7 @@
##############################################################################
"""Bootstrap the buildout project itself.
This is different from a normal boostrapping process because the
This is different from a normal boo
t
strapping process because the
buildout egg itself is installed as a develop egg.
$Id$
...
...
setup.py
View file @
6107db1f
...
...
@@ -12,7 +12,7 @@
#
##############################################################################
name
=
"zc.buildout"
version
=
"1.4.
0
dev"
version
=
"1.4.
4
dev"
import
os
from
setuptools
import
setup
...
...
@@ -48,6 +48,8 @@ long_description=(
+
'
\
n
'
+
read
(
'src'
,
'zc'
,
'buildout'
,
'easy_install.txt'
)
+
'
\
n
'
+
read
(
'src'
,
'zc'
,
'buildout'
,
'distribute.txt'
)
+
'
\
n
'
+
read
(
'CHANGES.txt'
)
+
'
\
n
'
+
'Download
\
n
'
...
...
@@ -72,7 +74,7 @@ setup(
long_description
=
long_description
,
license
=
"ZPL 2.1"
,
keywords
=
"development build"
,
url
=
'http://
pypi.python.org/pypi/zc.buildout
'
,
url
=
'http://
buildout.org
'
,
data_files
=
[(
'.'
,
[
'README.txt'
])],
packages
=
[
'zc'
,
'zc.buildout'
],
...
...
src/zc/buildout/bootstrap.txt
View file @
6107db1f
...
...
@@ -119,9 +119,86 @@ Let's make sure the generated `buildout` script uses it::
zc.buildout.buildout.main()
<BLANKLINE>
You can specify a location of ez_setup.py, so you can rely on a local or remote
location. We'll write our own ez_setup.py that we will also use to test some
other bootstrap options.
`zc.buildout` now can also run with `Distribute` with the `--distribute`
option::
>>> print 'X'; print system(
... zc.buildout.easy_install._safe_arg(sys.executable)+' '+
... 'bootstrap.py --distribute'); print 'X' # doctest: +ELLIPSIS
...
X
...
Generated script '/sample/bin/buildout'.
<BLANKLINE>
X
Let's make sure the generated `buildout` script uses it::
>>> print open(buildout_script).read() # doctest: +ELLIPSIS
#...
<BLANKLINE>
import sys
sys.path[0:0] = [
'/sample/eggs/distribute-...egg',
'/sample/eggs/zc.buildout-...egg',
]
<BLANKLINE>
import zc.buildout.buildout
<BLANKLINE>
if __name__ == '__main__':
zc.buildout.buildout.main()
<BLANKLINE>
Make sure both options can be used together::
>>> print 'X'; print system(
... zc.buildout.easy_install._safe_arg(sys.executable)+' '+
... 'bootstrap.py --distribute --version 1.2.1'); print 'X'
... # doctest: +ELLIPSIS
...
X
...
Generated script '/sample/bin/buildout'.
<BLANKLINE>
X
Let's make sure the generated `buildout` script uses ``Distribute`` *and*
``zc.buildout-1.2.1``::
>>> print open(buildout_script).read() # doctest: +ELLIPSIS
#...
<BLANKLINE>
import sys
sys.path[0:0] = [
'/sample/eggs/distribute-...egg',
'/sample/eggs/zc.buildout-1.2.1...egg',
]
<BLANKLINE>
import zc.buildout.buildout
<BLANKLINE>
if __name__ == '__main__':
zc.buildout.buildout.main()
<BLANKLINE>
Last, the -c option needs to work on bootstrap.py::
>>> conf_file = os.path.join(sample_buildout, 'other.cfg')
>>> f = open(conf_file, 'w')
>>> f.write('[buildout]\nparts=\n\n')
>>> f.close()
>>> print 'X'; print system(
... zc.buildout.easy_install._safe_arg(sys.executable)+' '+
... 'bootstrap.py -c %s --distribute' % conf_file); print 'X' # doctest: +ELLIPSIS
...
X
...
Generated script '/sample/bin/buildout'.
<BLANKLINE>
X
You can specify a location of ez_setup.py or distribute_setup, so you
can rely on a local or remote location. We'll write our own ez_setup.py
that we will also use to test some other bootstrap options.
>>> write('ez_setup.py', '''\
... def use_setuptools(**kwargs):
...
...
@@ -131,7 +208,7 @@ other bootstrap options.
... ''')
>>> print system(
... zc.buildout.easy_install._safe_arg(sys.executable)+' '+
... 'bootstrap.py --
ez_
setup-source=./ez_setup.py')
... 'bootstrap.py --setup-source=./ez_setup.py')
... # doctest: +ELLIPSIS
{'download_delay': 0,
'to_dir': '...'}
...
...
@@ -142,7 +219,7 @@ You can also pass a download-cache, and a place in which eggs should be stored
>>> print system(
... zc.buildout.easy_install._safe_arg(sys.executable)+' '+
... 'bootstrap.py --
ez_
setup-source=./ez_setup.py '+
... 'bootstrap.py --setup-source=./ez_setup.py '+
... '--download-base=./download-cache --eggs=eggs')
... # doctest: +ELLIPSIS
{'download_base': '/sample/download-cache/',
...
...
@@ -156,34 +233,33 @@ Here's the entire help text.
... zc.buildout.easy_install._safe_arg(sys.executable)+' '+
... 'bootstrap.py --help'),
... # doctest: +ELLIPSIS
Bootstrap a buildout-based project
usage: [DESIRED PYTHON FOR BUILDOUT] bootstrap.py [options]
<BLANKLINE>
Simply run this script in a directory containing a buildout.cfg.
The script accepts buildout command-line options, so you can
use the -c option to specify an alternate configuration file.
Bootstraps a buildout-based project.
<BLANKLINE>
...
Simply run this script in a directory containing a buildout.cfg, using the
Python that you want bin/buildout to use.
<BLANKLINE>
Note that by using --setup-source and --download-base to point to
local resources, you can keep this script from going over the network.
<BLANKLINE>
This script recognizes the following options itself. The first option it
encounters that is not one of these will cause the script to stop parsing
options and pass the rest on to buildout. Therefore, if you want to use
any of the following options *and* buildout command-line options like
-c, first use the following options, and then use the buildout options.
<BLANKLINE>
Options:
--version=ZC_BUILDOUT_VERSION
Specify a version number of the zc.buildout to use
--ez_setup-source=URL_OR_FILE
Specify a URL or file location for the ez_setup file.
Defaults to
http://peak.telecommunity.com/dist/ez_setup.py
--download-base=URL_OR_DIRECTORY
Specify a URL or directory for downloading setuptools and
zc.buildout. Defaults to PyPI.
--eggs=DIRECTORY
Specify a directory for storing eggs. Defaults to a temporary
directory that is deleted when the bootstrap script completes.
<BLANKLINE>
By using --ez_setup-source and --download-base to point to local resources,
you can keep this script from going over the network.
<BLANKLINE>
options:
-h, --help show this help message and exit
-v VERSION, --version=VERSION
use a specific zc.buildout version
-d, --distribute Use Distribute rather than Setuptools.
--setup-source=SETUP_SOURCE
Specify a URL or file location for the setup file. If
you use Setuptools, this will default to
http://peak.telecommunity.com/dist/ez_setup.py; if you
use Distribute, this will default to http://python-
distribute.org/distribute_setup.py.
--download-base=DOWNLOAD_BASE
Specify a URL or directory for downloading zc.buildout
and either Setuptools or Distribute. Defaults to PyPI.
--eggs=EGGS Specify a directory for storing eggs. Defaults to a
temporary directory that is deleted when the bootstrap
script completes.
-c CONFIG_FILE Specify the path to the buildout configuration file to
be used.
src/zc/buildout/buildout.py
View file @
6107db1f
...
...
@@ -837,7 +837,8 @@ class Buildout(UserDict.DictMixin):
upgraded
=
[]
for
project
in
'zc.buildout'
,
'setuptools'
:
req
=
pkg_resources
.
Requirement
.
parse
(
project
)
if
ws
.
find
(
req
)
!=
pkg_resources
.
working_set
.
find
(
req
):
project_location
=
pkg_resources
.
working_set
.
find
(
req
).
location
if
ws
.
find
(
req
).
location
!=
project_location
:
upgraded
.
append
(
ws
.
find
(
req
))
if
not
upgraded
:
...
...
src/zc/buildout/distribute.txt
0 → 100644
View file @
6107db1f
Distribute Support
==================
Distribute is a drop-in replacement for Setuptools.
zc.buildout is now compatible with Distribute 0.6. To use Distribute in your
buildout, you need use the ``--distribute`` option of the ``bootstrap.py``
script::
$ python bootstrap.py --distribute
This will download and install the latest Distribute 0.6 release in the
``eggs`` directory, and use this version for the scripts that are created
in ``bin``.
Notice that if you have a shared eggs directory, a buildout that uses
Distribute will not interfer with other buildouts that are based on Setuptools
and that are sharing the same eggs directory.
Form more information about the Distribute project, see:
http://python-distribute.org
src/zc/buildout/easy_install.py
View file @
6107db1f
...
...
@@ -137,50 +137,9 @@ if is_win32:
else:
_safe_arg = str
# The following string is used to run easy_install in
# Installer._call_easy_install. It is started with python -S (that is,
# don'
t
import
site
at
start
).
That
flag
,
and
all
of
the
code
in
this
# snippet above the last two lines, exist to work around a relatively rare
# problem. If
#
# - your buildout configuration is trying to install a package that is within
# a namespace package, and
#
# - you use a Python that has a different version of this package
# installed in in its site-packages using
# --single-version-externally-managed (that is, using the mechanism
# sometimes used by system packagers:
# http://peak.telecommunity.com/DevCenter/setuptools#install-command ), and
#
# - the new package tries to do sys.path tricks in the setup.py to get a
# __version__,
#
# then the older package will be loaded first, making the setup version
# the wrong number. While very arguably packages simply shouldn't do
# the sys.path tricks, some do, and we don't want buildout to fall over
# when they do.
#
# The namespace packages installed in site-packages with
# --single-version-externally-managed use a mechanism that cause them to
# be processed when site.py is imported. Simply starting Python with -S
# addresses the problem in Python 2.4 and 2.5, but Python 2.6's distutils
# imports a value from the site module, so we unfortunately have to do more
# drastic surgery in the _easy_install_cmd code below. The changes to
# sys.modules specifically try to only remove namespace modules installed by
# the --single-version-externally-managed code.
_easy_install_cmd
=
_safe_arg
(
'''
\
import sys;
\
p = sys.path[:];
\
m = sys.modules.keys();
\
import site;
\
sys.path[:] = p;
\
m_attrs = set(('__builtins__', '__file__', '__package__', '__path__'));
\
match = set(('__path__',));
\
[sys.modules.pop(k) for k, v in sys.modules.items()
\
if k not in m and v and m_attrs.intersection(dir(v)) == match];
\
from setuptools.command.easy_install import main;
\
main()'''
)
_easy_install_cmd = _safe_arg(
'
from
setuptools.command.easy_install
import
main
;
main
()
'
)
class Installer:
...
...
@@ -340,12 +299,9 @@ class Installer:
tmp
=
tempfile
.
mkdtemp
(
dir
=
dest
)
try
:
path
=
self
.
_get_dist
(
self
.
_constrain
(
pkg_resources
.
Requirement
.
parse
(
'setuptools'
)),
ws
,
False
,
)[
0
].
location
path
=
setuptools_loc
args
=
(
'-
S
c'
,
_easy_install_cmd
,
'-mUNxd'
,
_safe_arg
(
tmp
))
args
=
(
'-c'
,
_easy_install_cmd
,
'-mUNxd'
,
_safe_arg
(
tmp
))
if
self
.
_always_unzip
:
args
+=
(
'-Z'
,
)
level
=
logger
.
getEffectiveLevel
()
...
...
@@ -386,8 +342,8 @@ class Installer:
if
exit_code
:
logger
.
error
(
"An error occured when trying to install %s."
"Look above this message for any errors that"
"An error occured when trying to install %s.
"
"Look above this message for any errors that
"
"were output by easy_install."
,
dist
)
...
...
@@ -539,7 +495,7 @@ class Installer:
if
dist
is
None
:
raise
zc
.
buildout
.
UserError
(
"Coul
d
n't download distribution %s."
%
avail
)
"Couln't download distribution %s."
%
avail
)
if
dist
.
precedence
==
pkg_resources
.
EGG_DIST
:
# It's already an egg, just fetch it into the dest
...
...
@@ -672,9 +628,9 @@ class Installer:
logger
.
debug
(
'Installing %s.'
,
repr
(
specs
)[
1
:
-
1
])
path
=
self
.
_path
dest
ination
=
self
.
_dest
if
dest
ination
is
not
None
and
destination
not
in
path
:
path
.
insert
(
0
,
dest
ination
)
dest
=
self
.
_dest
if
dest
is
not
None
and
dest
not
in
path
:
path
.
insert
(
0
,
dest
)
requirements
=
[
self
.
_constrain
(
pkg_resources
.
Requirement
.
parse
(
spec
))
for
spec
in
specs
]
...
...
@@ -705,7 +661,7 @@ class Installer:
except
pkg_resources
.
DistributionNotFound
,
err
:
[
requirement
]
=
err
requirement
=
self
.
_constrain
(
requirement
)
if
dest
ination
:
if
dest
:
logger
.
debug
(
'Getting required %r'
,
str
(
requirement
))
else
:
logger
.
debug
(
'Adding required %r'
,
str
(
requirement
))
...
...
@@ -948,9 +904,6 @@ def develop(setup, dest,
def
working_set
(
specs
,
executable
,
path
):
return
install
(
specs
,
None
,
executable
=
executable
,
path
=
path
)
############################################################################
# Script generation functions
def
scripts
(
reqs
,
working_set
,
executable
,
dest
,
scripts
=
None
,
extra_paths
=
(),
...
...
@@ -959,84 +912,20 @@ def scripts(reqs, working_set, executable, dest,
initialization
=
''
,
relative_paths
=
False
,
):
"""Generate scripts and/or an interpreter.
See generate_scripts for a newer version with more options and a
different approach.
"""
path
=
_get_path
(
working_set
,
extra_paths
)
if
initialization
:
initialization
=
'
\
n
'
+
initialization
+
'
\
n
'
generated
=
_generate_scripts
(
reqs
,
working_set
,
dest
,
path
,
scripts
,
relative_paths
,
initialization
,
executable
,
arguments
)
if
interpreter
:
sname
=
os
.
path
.
join
(
dest
,
interpreter
)
spath
,
rpsetup
=
_relative_path_and_setup
(
sname
,
path
,
relative_paths
)
generated
.
extend
(
_pyscript
(
spath
,
sname
,
executable
,
rpsetup
))
return
generated
path
=
[
dist
.
location
for
dist
in
working_set
]
path
.
extend
(
extra_paths
)
path
=
map
(
realpath
,
path
)
def
generate_scripts
(
dest
,
working_set
,
executable
,
site_py_dest
,
reqs
=
(),
scripts
=
None
,
interpreter
=
None
,
extra_paths
=
(),
initialization
=
''
,
add_site_packages
=
False
,
exec_sitecustomize
=
False
,
relative_paths
=
False
,
script_arguments
=
''
,
script_initialization
=
''
):
"""Generate scripts and/or an interpreter.
This accomplishes the same job as the ``scripts`` function, above,
but it does so in an alternative way that allows safely including
Python site packages, if desired, and choosing to execute the Python's
sitecustomize.
"""
generated
=
[]
generated
.
append
(
_generate_sitecustomize
(
site_py_dest
,
executable
,
initialization
,
exec_sitecustomize
))
generated
.
append
(
_generate_site
(
site_py_dest
,
working_set
,
executable
,
extra_paths
,
add_site_packages
,
relative_paths
))
script_initialization
=
(
'
\
n
import site # imports custom buildbot-generated site.py
\
n
%s'
%
(
script_initialization
,))
if
not
script_initialization
.
endswith
(
'
\
n
'
):
script_initialization
+=
'
\
n
'
generated
.
extend
(
_generate_scripts
(
reqs
,
working_set
,
dest
,
[
site_py_dest
],
scripts
,
relative_paths
,
script_initialization
,
executable
,
script_arguments
))
if
interpreter
:
generated
.
extend
(
_generate_interpreter
(
interpreter
,
dest
,
executable
,
site_py_dest
,
relative_paths
))
return
generated
# Utilities for the script generation functions.
# These are shared by both ``scripts`` and ``generate_scripts``
def
_get_path
(
working_set
,
extra_paths
=
()):
"""Given working set and extra paths, return a normalized path list."""
path
=
[
dist
.
location
for
dist
in
working_set
]
path
.
extend
(
extra_paths
)
return
map
(
realpath
,
path
)
def
_generate_scripts
(
reqs
,
working_set
,
dest
,
path
,
scripts
,
relative_paths
,
initialization
,
executable
,
arguments
):
"""Generate scripts for the given requirements.
- reqs is an iterable of string requirements or entry points.
- The requirements must be findable in the given working_set.
- The dest is the directory in which the scripts should be created.
- The path is a list of paths that should be added to sys.path.
- The scripts is an optional dictionary. If included, the keys should be
the names of the scripts that should be created, as identified in their
entry points; and the values should be the name the script should
actually be created with.
- relative_paths, if given, should be the path that is the root of the
buildout (the common path that should be the root of what is relative).
"""
if
isinstance
(
reqs
,
str
):
raise
TypeError
(
'Expected iterable of requirements or entry points,'
' got string.'
)
generated
=
[]
if
initialization
:
initialization
=
'
\
n
'
+
initialization
+
'
\
n
'
entry_points
=
[]
for
req
in
reqs
:
if
isinstance
(
req
,
str
):
...
...
@@ -1050,6 +939,7 @@ def _generate_scripts(reqs, working_set, dest, path, scripts, relative_paths,
)
else
:
entry_points
.
append
(
req
)
for
name
,
module_name
,
attrs
in
entry_points
:
if
scripts
is
not
None
:
sname
=
scripts
.
get
(
name
)
...
...
@@ -1057,48 +947,40 @@ def _generate_scripts(reqs, working_set, dest, path, scripts, relative_paths,
continue
else
:
sname
=
name
sname
=
os
.
path
.
join
(
dest
,
sname
)
spath
,
rpsetup
=
_relative_path_and_setup
(
sname
,
path
,
relative_paths
)
generated
.
extend
(
_script
(
sname
,
executable
,
rpsetup
,
spath
,
initialization
,
module_name
,
attrs
,
arguments
)
)
return
generated
_script
(
module_name
,
attrs
,
spath
,
sname
,
executable
,
arguments
,
initialization
,
rpsetup
)
)
def
_relative_path_and_setup
(
sname
,
path
,
relative_paths
=
False
,
indent_level
=
1
):
"""Return a string of code of paths and of setup if appropriate.
if
interpreter
:
sname
=
os
.
path
.
join
(
dest
,
interpreter
)
spath
,
rpsetup
=
_relative_path_and_setup
(
sname
,
path
,
relative_paths
)
generated
.
extend
(
_pyscript
(
spath
,
sname
,
executable
,
rpsetup
))
- sname is the full path to the script name to be created.
- path is the list of paths to be added to sys.path.
- relative_paths, if given, should be the path that is the root of the
buildout (the common path that should be the root of what is relative).
- indent_level is the number of four-space indents that the path should
insert before each element of the path.
"""
return
generated
def
_relative_path_and_setup
(
sname
,
path
,
relative_paths
):
if
relative_paths
:
relative_paths
=
os
.
path
.
normcase
(
relative_paths
)
sname
=
os
.
path
.
normcase
(
os
.
path
.
abspath
(
sname
))
spath
=
_format_paths
(
spath
=
',
\
n
'
.
join
(
[
_relativitize
(
os
.
path
.
normcase
(
path_item
),
sname
,
relative_paths
)
for
path_item
in
path
],
indent_level
=
indent_level
)
for
path_item
in
path
]
)
rpsetup
=
relative_paths_setup
for
i
in
range
(
_relative_depth
(
relative_paths
,
sname
)):
rpsetup
+=
"base = os.path.dirname(base)
\
n
"
else
:
spath
=
_format_paths
((
repr
(
p
)
for
p
in
path
),
indent_level
=
indent_level
)
spath
=
repr
(
path
)[
1
:
-
1
].
replace
(
', '
,
',
\
n
'
)
rpsetup
=
''
return
spath
,
rpsetup
def
_relative_depth
(
common
,
path
):
"""Return number of dirs separating ``path`` from ancestor, ``common``.
For instance, if path is /foo/bar/baz/bing, and common is /foo, this will
return 2--in UNIX, the number of ".." to get from bing's directory
to foo.
This is a helper for _relative_path_and_setup.
"""
def
_relative_depth
(
common
,
path
):
n
=
0
while
1
:
dirname
=
os
.
path
.
dirname
(
path
)
...
...
@@ -1111,11 +993,6 @@ def _relative_depth(common, path):
return
n
def
_relative_path
(
common
,
path
):
"""Return the relative path from ``common`` to ``path``.
This is a helper for _relativitize, which is a helper to
_relative_path_and_setup.
"""
r
=
[]
while
1
:
dirname
,
basename
=
os
.
path
.
split
(
path
)
...
...
@@ -1129,11 +1006,6 @@ def _relative_path(common, path):
return
os
.
path
.
join
(
*
r
)
def
_relativitize
(
path
,
script
,
relative_paths
):
"""Return a code string for the given path.
Path is relative to the base path ``relative_paths``if the common prefix
between ``path`` and ``script`` starts with ``relative_paths``.
"""
if
path
==
script
:
raise
AssertionError
(
"path == script"
)
common
=
os
.
path
.
dirname
(
os
.
path
.
commonprefix
([
path
,
script
]))
...
...
@@ -1144,6 +1016,7 @@ def _relativitize(path, script, relative_paths):
else
:
return
repr
(
path
)
relative_paths_setup
=
"""
import os
...
...
@@ -1151,73 +1024,58 @@ join = os.path.join
base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
"""
def
_write_script
(
full_name
,
contents
,
logged_type
):
"""Write contents of script in full_name, logging the action.
The only tricky bit in this function is that it supports Windows by
creating exe files using a pkg_resources helper.
"""
def
_script
(
module_name
,
attrs
,
path
,
dest
,
executable
,
arguments
,
initialization
,
rsetup
):
generated
=
[]
script_name
=
full_name
script
=
dest
if
is_win32
:
dest
+=
'-script.py'
contents
=
script_template
%
dict
(
python
=
_safe_arg
(
executable
),
path
=
path
,
module_name
=
module_name
,
attrs
=
attrs
,
arguments
=
arguments
,
initialization
=
initialization
,
relative_paths_setup
=
rsetup
,
)
changed
=
not
(
os
.
path
.
exists
(
dest
)
and
open
(
dest
).
read
()
==
contents
)
if
is_win32
:
script_name
+=
'-script.py'
# Generate exe file and give the script a magic name.
exe
=
full_name
+
'.exe'
# generate exe file and give the script a magic name:
exe
=
script
+
'.exe'
new_data
=
pkg_resources
.
resource_string
(
'setuptools'
,
'cli.exe'
)
if
not
os
.
path
.
exists
(
exe
)
or
(
open
(
exe
,
'rb'
).
read
()
!=
new_data
):
# Only write it if it's different.
open
(
exe
,
'wb'
).
write
(
new_data
)
generated
.
append
(
exe
)
changed
=
not
(
os
.
path
.
exists
(
script_name
)
and
open
(
script_name
).
read
()
==
contents
)
if
changed
:
open
(
script_name
,
'w'
).
write
(
contents
)
open
(
dest
,
'w'
).
write
(
contents
)
logger
.
info
(
"Generated script %r."
,
script
)
try
:
os
.
chmod
(
script_name
,
0755
)
os
.
chmod
(
dest
,
0755
)
except
(
AttributeError
,
os
.
error
):
pass
logger
.
info
(
"Generated %s %r."
,
logged_type
,
full_name
)
generated
.
append
(
script_name
)
return
generated
def
_format_paths
(
paths
,
indent_level
=
1
):
"""Format paths for inclusion in a script."""
separator
=
',
\
n
'
+
indent_level
*
' '
return
separator
.
join
(
paths
)
def
_script
(
dest
,
executable
,
relative_paths_setup
,
path
,
initialization
,
module_name
,
attrs
,
arguments
):
contents
=
script_template
%
dict
(
python
=
_safe_arg
(
executable
),
path
=
path
,
module_name
=
module_name
,
attrs
=
attrs
,
arguments
=
arguments
,
initialization
=
initialization
,
relative_paths_setup
=
relative_paths_setup
,
)
return
_write_script
(
dest
,
contents
,
'script'
)
generated
.
append
(
dest
)
return
generated
if
is_jython
and
jython_os_name
==
'linux'
:
script_header
=
'#!/usr/bin/env %(python)s
-S
'
script_header
=
'#!/usr/bin/env %(python)s'
else
:
script_header
=
'#!%(python)s
-S
'
script_header
=
'#!%(python)s'
sys_path_template
=
'''
\
import sys
sys.path[0:0] = [
%s,
]
'''
script_template
=
script_header
+
'''
\
%(relative_paths_setup)s
import sys
sys.path[0:0] = [
%(path)s,
]
%(path)s,
]
%(initialization)s
import %(module_name)s
...
...
@@ -1225,15 +1083,38 @@ if __name__ == '__main__':
%(module_name)s.%(attrs)s(%(arguments)s)
'''
# These are used only by the older ``scripts`` function.
def
_pyscript
(
path
,
dest
,
executable
,
rsetup
):
generated
=
[]
script
=
dest
if
is_win32
:
dest
+=
'-script.py'
contents
=
py_script_template
%
dict
(
python
=
_safe_arg
(
executable
),
path
=
path
,
relative_paths_setup
=
rsetup
,
)
return
_write_script
(
dest
,
contents
,
'interpreter'
)
changed
=
not
(
os
.
path
.
exists
(
dest
)
and
open
(
dest
).
read
()
==
contents
)
if
is_win32
:
# generate exe file and give the script a magic name:
exe
=
script
+
'.exe'
open
(
exe
,
'wb'
).
write
(
pkg_resources
.
resource_string
(
'setuptools'
,
'cli.exe'
)
)
generated
.
append
(
exe
)
if
changed
:
open
(
dest
,
'w'
).
write
(
contents
)
try
:
os
.
chmod
(
dest
,
0755
)
except
(
AttributeError
,
os
.
error
):
pass
logger
.
info
(
"Generated interpreter %r."
,
script
)
generated
.
append
(
dest
)
return
generated
py_script_template
=
script_header
+
'''
\
...
...
@@ -1241,8 +1122,8 @@ py_script_template = script_header + '''\
import sys
sys.path[0:0] = [
%(path)s,
]
%(path)s,
]
_interactive = True
if len(sys.argv) > 1:
...
...
@@ -1270,312 +1151,6 @@ if _interactive:
__import__("code").interact(banner="", local=globals())
'''
# These are used only by the newer ``generate_scripts`` function.
def
_get_system_paths
(
executable
):
"""return lists of standard lib and site paths for executable.
"""
# We want to get a list of the site packages, which is not easy.
# The canonical way to do this is to use
# distutils.sysconfig.get_python_lib(), but that only returns a
# single path, which does not reflect reality for many system
# Pythons, which have multiple additions. Instead, we start Python
# with -S, which does not import site.py and set up the extra paths
# like site-packages or (Ubuntu/Debian) dist-packages and
# python-support. We then compare that sys.path with the normal one
# (minus user packages if this is Python 2.6, because we don't
# support those (yet?). The set of the normal one minus the set of
# the ones in ``python -S`` is the set of packages that are
# effectively site-packages.
#
# The given executable might not be the current executable, so it is
# appropriate to do another subprocess to figure out what the
# additional site-package paths are. Moreover, even if this
# executable *is* the current executable, this code might be run in
# the context of code that has manipulated the sys.path--for
# instance, to add local zc.buildout or setuptools eggs.
def
get_sys_path
(
*
args
,
**
kwargs
):
cmd
=
[
executable
]
cmd
.
extend
(
args
)
cmd
.
extend
([
"-c"
,
"import sys, os;"
"print repr([os.path.normpath(p) for p in sys.path if p])"
])
# Windows needs some (as yet to be determined) part of the real env.
env
=
os
.
environ
.
copy
()
env
.
update
(
kwargs
)
_proc
=
subprocess
.
Popen
(
cmd
,
stdout
=
subprocess
.
PIPE
,
stderr
=
subprocess
.
PIPE
,
env
=
env
)
stdout
,
stderr
=
_proc
.
communicate
();
if
_proc
.
returncode
:
raise
RuntimeError
(
'error trying to get system packages:
\
n
%s'
%
(
stderr
,))
res
=
eval
(
stdout
.
strip
())
try
:
res
.
remove
(
'.'
)
except
ValueError
:
pass
return
res
stdlib
=
get_sys_path
(
'-S'
)
# stdlib only
no_user_paths
=
get_sys_path
(
PYTHONNOUSERSITE
=
'x'
)
site_paths
=
[
p
for
p
in
no_user_paths
if
p
not
in
stdlib
]
return
(
stdlib
,
site_paths
)
def
_get_module_file
(
executable
,
name
):
"""Return a module's file path.
- executable is a path to the desired Python executable.
- name is the name of the (pure, not C) Python module.
"""
cmd
=
[
executable
,
"-c"
,
"import imp; "
"fp, path, desc = imp.find_module(%r); "
"fp.close; "
"print path"
%
(
name
,)]
_proc
=
subprocess
.
Popen
(
cmd
,
stdout
=
subprocess
.
PIPE
,
stderr
=
subprocess
.
PIPE
)
stdout
,
stderr
=
_proc
.
communicate
();
if
_proc
.
returncode
:
logger
.
info
(
'Could not find file for module %s:
\
n
%s'
,
name
,
stderr
)
return
None
# else: ...
res
=
stdout
.
strip
()
if
res
.
endswith
(
'.pyc'
)
or
res
.
endswith
(
'.pyo'
):
raise
RuntimeError
(
'Cannot find uncompiled version of %s'
%
(
name
,))
if
not
os
.
path
.
exists
(
res
):
raise
RuntimeError
(
'File does not exist for module %s:
\
n
%s'
%
(
name
,
res
))
return
res
def
_generate_sitecustomize
(
dest
,
executable
,
initialization
=
''
,
exec_sitecustomize
=
False
):
"""Write a sitecustomize file with optional custom initialization.
The created script will execute the underlying Python's
sitecustomize if exec_sitecustomize is True.
"""
sitecustomize_path
=
os
.
path
.
join
(
dest
,
'sitecustomize.py'
)
sitecustomize
=
open
(
sitecustomize_path
,
'w'
)
if
initialization
:
sitecustomize
.
write
(
initialization
+
'
\
n
'
)
if
exec_sitecustomize
:
real_sitecustomize_path
=
_get_module_file
(
executable
,
'sitecustomize'
)
if
real_sitecustomize_path
:
real_sitecustomize
=
open
(
real_sitecustomize_path
,
'r'
)
sitecustomize
.
write
(
'
\
n
# The following is from
\
n
# %s
\
n
'
%
(
real_sitecustomize_path
,))
sitecustomize
.
write
(
real_sitecustomize
.
read
())
real_sitecustomize
.
close
()
sitecustomize
.
close
()
return
sitecustomize_path
def
_generate_site
(
dest
,
working_set
,
executable
,
extra_paths
=
(),
add_site_packages
=
False
,
relative_paths
=
False
):
"""Write a site.py file with eggs from working_set.
extra_paths will be added to the path. If add_site_packages is True,
paths from the underlying Python will be added.
"""
path
=
_get_path
(
working_set
,
extra_paths
)
site_path
=
os
.
path
.
join
(
dest
,
'site.py'
)
path_string
,
rpsetup
=
_relative_path_and_setup
(
site_path
,
path
,
relative_paths
,
indent_level
=
2
)
if
rpsetup
:
rpsetup
=
'
\
n
'
.
join
(
[(
line
and
' %s'
%
(
line
,)
or
line
)
for
line
in
rpsetup
.
split
(
'
\
n
'
)])
real_site_path
=
_get_module_file
(
executable
,
'site'
)
real_site
=
open
(
real_site_path
,
'r'
)
site
=
open
(
site_path
,
'w'
)
extra_path_snippet
=
add_site_packages_snippet
[
add_site_packages
]
extra_path_snippet_followup
=
add_site_packages_snippet_followup
[
add_site_packages
]
if
add_site_packages
:
stdlib
,
site_paths
=
_get_system_paths
(
executable
)
extra_path_snippet
=
extra_path_snippet
%
_format_paths
(
(
repr
(
p
)
for
p
in
site_paths
),
2
)
addsitepackages_marker
=
'def addsitepackages('
enableusersite_marker
=
'ENABLE_USER_SITE = '
successful_rewrite
=
False
for
line
in
real_site
.
readlines
():
if
line
.
startswith
(
enableusersite_marker
):
site
.
write
(
enableusersite_marker
)
site
.
write
(
'False # buildout does not support user sites.
\
n
'
)
elif
line
.
startswith
(
addsitepackages_marker
):
site
.
write
(
addsitepackages_script
%
(
extra_path_snippet
,
rpsetup
,
path_string
,
extra_path_snippet_followup
))
site
.
write
(
line
[
len
(
addsitepackages_marker
):])
successful_rewrite
=
True
else
:
site
.
write
(
line
)
if
not
successful_rewrite
:
raise
RuntimeError
(
'Buildout did not successfully rewrite site.py'
)
return
site_path
add_site_packages_snippet
=
[
'''
paths = []'''
,
'''
paths = [ # These are the underlying Python's site-packages.
%s]
sys.path[0:0] = paths
known_paths.update([os.path.normcase(os.path.abspath(p)) for p in paths])
try:
import pkg_resources
except ImportError:
# No namespace packages in sys.path; no fixup needed.
pkg_resources = None'''
]
add_site_packages_snippet_followup
=
[
''
,
'''
if pkg_resources is not None:
# There may be namespace packages in sys.path. This is much faster
# than importing pkg_resources after the sys.path has a large number
# of eggs.
for p in sys.path:
pkg_resources.fixup_namespace_packages(p)'''
]
addsitepackages_script
=
'''
\
def addsitepackages(known_paths):%s
%s paths[0:0] = [ # eggs
%s
]
# Process all dirs. Look for .pth files. If they exist, defer
# processing "import" varieties.
dotpth = os.extsep + "pth"
deferred = []
for path in reversed(paths):
# Duplicating addsitedir.
sitedir, sitedircase = makepath(path)
if not sitedircase in known_paths and os.path.exists(sitedir):
sys.path.insert(0, sitedir)
known_paths.add(sitedircase)
try:
names = os.listdir(sitedir)
except os.error:
continue
names = [name for name in names if name.endswith(dotpth)]
names.sort()
for name in names:
# Duplicating addpackage.
fullname = os.path.join(sitedir, name)
try:
f = open(fullname, "rU")
except IOError:
continue
try:
for line in f:
if line.startswith("#"):
continue
if (line.startswith("import ") or
line.startswith("import
\
t
")):
# This line is supposed to be executed. It
# might be a setuptools namespace package
# installed with a system package manager.
# Defer this so we can process egg namespace
# packages first, or else the eggs with the same
# namespace will be ignored.
deferred.append((sitedir, name, fullname, line))
continue
line = line.rstrip()
dir, dircase = makepath(sitedir, line)
if not dircase in known_paths and os.path.exists(dir):
sys.path.append(dir)
known_paths.add(dircase)
finally:
f.close()%s
# Process "import ..." .pth lines.
for sitedir, name, fullname, line in deferred:
# Note that some lines--such as the one setuptools writes for
# namespace packages--expect some or all of sitedir, name, and
# fullname to be present in the frame locals, as it is in
# ``addpackage``.
try:
exec line
except:
print "Error in %%s" %% (fullname,)
raise
global addsitepackages
addsitepackages = original_addsitepackages
return known_paths
buildout_addsitepackages = addsitepackages
def original_addsitepackages('''
def
_generate_interpreter
(
name
,
dest
,
executable
,
site_py_dest
,
relative_paths
=
False
):
"""Write an interpreter script, using the site.py approach."""
full_name
=
os
.
path
.
join
(
dest
,
name
)
site_py_dest_string
,
rpsetup
=
_relative_path_and_setup
(
full_name
,
[
site_py_dest
],
relative_paths
)
if
sys
.
platform
==
'win32'
:
windows_import
=
'
\
n
import subprocess'
# os.exec* is a mess on Windows, particularly if the path
# to the executable has spaces and the Python is using MSVCRT.
# The standard fix is to surround the executable's path with quotes,
# but that has been unreliable in testing.
#
# Here's a demonstration of the problem. Given a Python
# compiled with a MSVCRT-based compiler, such as the free Visual
# C++ 2008 Express Edition, and an executable path with spaces
# in it such as the below, we see the following.
#
# >>> import os
# >>> p0 = 'C:\\Documents and Settings\\Administrator\\My Documents\\Downloads\\Python-2.6.4\\PCbuild\\python.exe'
# >>> os.path.exists(p0)
# True
# >>> os.execv(p0, [])
# Traceback (most recent call last):
# File "<stdin>", line 1, in <module>
# OSError: [Errno 22] Invalid argument
#
# That seems like a standard problem. The standard solution is
# to quote the path (see, for instance
# http://bugs.python.org/issue436259). However, this solution,
# and other variations, fail:
#
# >>> p1 = '"C:\\Documents and Settings\\Administrator\\My Documents\\Downloads\\Python-2.6.4\\PCbuild\\python.exe"'
# >>> os.execv(p1, [])
# Traceback (most recent call last):
# File "<stdin>", line 1, in <module>
# OSError: [Errno 22] Invalid argument
#
# We simply use subprocess instead, since it handles everything
# nicely, and the transparency of exec* (that is, not running,
# perhaps unexpectedly, in a subprocess) is arguably not a
# necessity, at least for many use cases.
execute
=
'subprocess.call(argv, env=environ)'
else
:
windows_import
=
''
execute
=
'os.execve(sys.executable, argv, environ)'
contents
=
interpreter_template
%
dict
(
python
=
_safe_arg
(
executable
),
site_dest
=
site_py_dest_string
,
relative_paths_setup
=
rpsetup
,
windows_import
=
windows_import
,
execute
=
execute
,
)
return
_write_script
(
full_name
,
contents
,
'interpreter'
)
interpreter_template
=
script_header
+
'''
\
%(relative_paths_setup)s
import os
import sys%(windows_import)s
argv = [sys.executable] + sys.argv[1:]
environ = os.environ.copy()
path = %(site_dest)s
if environ.get('PYTHONPATH'):
path = os.pathsep.join([path, environ['PYTHONPATH']])
environ['PYTHONPATH'] = path
%(execute)s
'''
# End of script generation code.
############################################################################
runsetup_template
=
"""
import sys
sys.path.insert(0, %(setupdir)r)
...
...
src/zc/buildout/easy_install.txt
View file @
6107db1f
...
...
@@ -521,30 +521,25 @@ To return the dependency_links behavior to normal call the function again.
Script generation
-----------------
The easy_install module provides support for creating scripts from
eggs.
It provides two competing functions. Both are similar to setuptools
except that they provides facilities for baking a script's path into the
script. This
has two advantages:
The easy_install module provides support for creating scripts from
eggs. It provides a function similar to setuptools except that it
provides facilities for baking a script's path into the script. This
has two advantages:
- The eggs to be used by a script are not chosen at run time, making
startup faster and, more importantly, deterministic.
- The script doesn't have to import pkg_resources because the logic that
pkg_resources would execute at run time is executed at script-creation
time. (There is an exception if you want to have your Python's site
packages available, as discussed below, but even in that case
pkg_resources is only partially activated.)
- The script doesn't have to import pkg_resources because the logic
that pkg_resources would execute at run time is executed at
script-creation time.
The ``scripts`` function
~~~~~~~~~~~~~~~~~~~~~~~~
The ``scripts`` function is the first way to generate scripts that we'll
examine. It is the earlier approach that the package offered. Let's
create a destination directory for it to place them in:
The scripts method can be used to generate scripts. Let's create a
destination directory for it to place them in:
>>> import tempfile
>>> bin = tmpdir('bin')
Now, we'll use the scripts
function
to generate scripts in this directory
Now, we'll use the scripts
method
to generate scripts in this directory
from the demo egg:
>>> import sys
...
...
@@ -586,7 +581,7 @@ interpreter and without having to provide a '.py' suffix.
The demo script run the entry point defined in the demo egg:
>>> cat(bin, 'demo') # doctest: +NORMALIZE_WHITESPACE
#!/usr/local/bin/python2.4
-S
#!/usr/local/bin/python2.4
<BLANKLINE>
import sys
sys.path[0:0] = [
...
...
@@ -606,15 +601,6 @@ Some things to note:
- The module for the script entry point is imported and the entry
point, in this case, 'main', is run.
- On the shebang (first) line of the script, Python is invoked with -S.
This means that site.py is not imported, which in turn means that
site-packages are not part of the path. This is the safest approach,
and let's you easily use a system Python to do buildout-based
development. Note that, because of the setuptools-provided .exe files
that buildout uses to run scripts, it also works on Windows. (If you
want to use site-packages, see the discussion--and warnings--for the
``generate_scripts`` function, below.)
Rather than requirement strings, you can pass tuples containing 3
strings:
...
...
@@ -632,7 +618,7 @@ rather than passing a requirement:
... ws, sys.executable, bin)
>>> cat(bin, 'demo') # doctest: +NORMALIZE_WHITESPACE
#!/usr/local/bin/python2.4
-S
#!/usr/local/bin/python2.4
<BLANKLINE>
import sys
sys.path[0:0] = [
...
...
@@ -676,7 +662,7 @@ The py script simply runs the Python interactive interpreter with
the path set:
>>> cat(bin, 'py') # doctest: +NORMALIZE_WHITESPACE
#!/usr/local/bin/python2.4
-S
#!/usr/local/bin/python2.4
<BLANKLINE>
import sys
<BLANKLINE>
...
...
@@ -750,8 +736,8 @@ original script names to new script names.
>>> print system(os.path.join(bin, 'run')),
3 1
The ``scripts`` function:
Including extra paths in scripts
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Including extra paths in scripts
--------------------------------
We can pass a keyword argument, extra paths, to cause additional paths
to be included in the a generated script:
...
...
@@ -762,7 +748,7 @@ to be included in the a generated script:
... extra_paths=[foo])
>>> cat(bin, 'run') # doctest: +NORMALIZE_WHITESPACE
#!/usr/local/bin/python2.4
-S
#!/usr/local/bin/python2.4
<BLANKLINE>
import sys
sys.path[0:0] = [
...
...
@@ -776,8 +762,8 @@ to be included in the a generated script:
if __name__ == '__main__':
eggrecipedemo.main()
The ``scripts`` function:
Providing script arguments
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Providing script arguments
--------------------------
An "argument" keyword argument can be used to pass arguments to an
entry point. The value passed is a source string to be placed between the
...
...
@@ -788,7 +774,7 @@ parentheses in the call:
... arguments='1, 2')
>>> cat(bin, 'run') # doctest: +NORMALIZE_WHITESPACE
#!/usr/local/bin/python2.4
-S
#!/usr/local/bin/python2.4
import sys
sys.path[0:0] = [
'/sample-install/demo-0.3-py2.4.egg',
...
...
@@ -800,8 +786,8 @@ parentheses in the call:
if __name__ == '__main__':
eggrecipedemo.main(1, 2)
The ``scripts`` function:
Passing initialization code
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Passing initialization code
---------------------------
You can also pass script initialization code:
...
...
@@ -811,7 +797,7 @@ You can also pass script initialization code:
... initialization='import os\nos.chdir("foo")')
>>> cat(bin, 'run') # doctest: +NORMALIZE_WHITESPACE
#!/usr/local/bin/python2.4
-S
#!/usr/local/bin/python2.4
import sys
sys.path[0:0] = [
'/sample-install/demo-0.3-py2.4.egg',
...
...
@@ -826,8 +812,8 @@ You can also pass script initialization code:
if __name__ == '__main__':
eggrecipedemo.main(1, 2)
The ``scripts`` function:
Relative paths
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Relative paths
--------------
Sometimes, you want to be able to move a buildout directory around and
have scripts still work without having to rebuild them. We can
...
...
@@ -851,7 +837,7 @@ to pass a common base directory of the scripts and eggs:
... relative_paths=bo)
>>> cat(bo, 'bin', 'run')
#!/usr/local/bin/python2.4
-S
#!/usr/local/bin/python2.4
<BLANKLINE>
import os
<BLANKLINE>
...
...
@@ -861,12 +847,11 @@ to pass a common base directory of the scripts and eggs:
<BLANKLINE>
import sys
sys.path[0:0] = [
join(base, 'eggs/demo-0.3-pyN.N.egg'),
join(base, 'eggs/demoneeded-1.1-pyN.N.egg'),
'/ba',
join(base, 'bar'),
]
<BLANKLINE>
join(base, 'eggs/demo-0.3-pyN.N.egg'),
join(base, 'eggs/demoneeded-1.1-pyN.N.egg'),
'/ba',
join(base, 'bar'),
]
<BLANKLINE>
import eggrecipedemo
<BLANKLINE>
...
...
@@ -884,7 +869,7 @@ Of course, running the script works:
We specified an interpreter and its paths are adjusted too:
>>> cat(bo, 'bin', 'py')
#!/usr/local/bin/python2.4
-S
#!/usr/local/bin/python2.4
<BLANKLINE>
import os
<BLANKLINE>
...
...
@@ -895,11 +880,11 @@ We specified an interpreter and its paths are adjusted too:
import sys
<BLANKLINE>
sys.path[0:0] = [
join(base, 'eggs/demo-0.3-pyN.N.egg'),
join(base, 'eggs/demoneeded-1.1-pyN.N.egg'),
'/ba',
join(base, 'bar'),
]
join(base, 'eggs/demo-0.3-pyN.N.egg'),
join(base, 'eggs/demoneeded-1.1-pyN.N.egg'),
'/ba',
join(base, 'bar'),
]
<BLANKLINE>
_interactive = True
if len(sys.argv) > 1:
...
...
@@ -926,551 +911,6 @@ We specified an interpreter and its paths are adjusted too:
del _interactive
__import__("code").interact(banner="", local=globals())
The ``generate_scripts`` function
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The newer function for creating scripts is ``generate_scripts``. It has the
same basic functionality as the ``scripts`` function: it can create scripts
to run arbitrary entry points, and to run a Python interpreter. The
following are the differences from a user's perspective.
- In contrast to the interpreter generated by the ``scripts`` method, which
supports only a small subset of the usual Python executable's options,
the interpreter generated by ``generate_scripts`` supports all of them.
This makes it possible to use as full Python replacement for scripts that
need the distributions specified in your buildout.
- Both the interpreter and the entry point scripts allow you to include the
site packages, and/or the sitecustomize, of the Python executable.
It works by creating site.py and sitecustomize.py files that set up the
desired paths and initialization. These must be placed within an otherwise
empty directory. Typically this is in a recipe's parts directory.
Here's the simplest example, building an interpreter script.
>>> interpreter_dir = tmpdir('interpreter')
>>> interpreter_parts_dir = os.path.join(
... interpreter_dir, 'parts', 'interpreter')
>>> interpreter_bin_dir = os.path.join(interpreter_dir, 'bin')
>>> mkdir(interpreter_bin_dir)
>>> mkdir(interpreter_dir, 'eggs')
>>> mkdir(interpreter_dir, 'parts')
>>> mkdir(interpreter_parts_dir)
>>> ws = zc.buildout.easy_install.install(
... ['demo'], join(interpreter_dir, 'eggs'), links=[link_server],
... index=link_server+'index/')
>>> generated = zc.buildout.easy_install.generate_scripts(
... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
... interpreter='py')
Depending on whether the machine being used is running Windows or not, this
produces either three or four files. In both cases, we have site.py and
sitecustomize.py generated in the parts/interpreter directory. For Windows,
we have py.exe and py-script.py; for other operating systems, we have py.
>>> sitecustomize_path = os.path.join(
... interpreter_parts_dir, 'sitecustomize.py')
>>> site_path = os.path.join(interpreter_parts_dir, 'site.py')
>>> interpreter_path = os.path.join(interpreter_bin_dir, 'py')
>>> if sys.platform == 'win32':
... py_path = os.path.join(interpreter_bin_dir, 'py-script.py')
... expected = [sitecustomize_path,
... site_path,
... os.path.join(interpreter_bin_dir, 'py.exe'),
... py_path]
... else:
... py_path = interpreter_path
... expected = [sitecustomize_path, site_path, py_path]
...
>>> assert generated == expected, repr((generated, expected))
We didn't ask for any initialization, and we didn't ask to use the underlying
sitecustomization, so sitecustomize.py is empty.
>>> cat(sitecustomize_path)
The interpreter script is simple. It puts the directory with the
site.py and sitecustomize.py on the PYTHONPATH and (re)starts Python.
>>> cat(py_path)
#!/usr/bin/python2.4 -S
<BLANKLINE>
import os
import sys
<BLANKLINE>
argv = [sys.executable] + sys.argv[1:]
environ = os.environ.copy()
path = '/interpreter/parts/interpreter'
if environ.get('PYTHONPATH'):
path = os.pathsep.join([path, environ['PYTHONPATH']])
environ['PYTHONPATH'] = path
os.execve(sys.executable, argv, environ)
The site.py file is a modified version of the underlying Python's site.py.
The most important modification is that it has a different version of the
addsitepackages function. It has all of the trickier bits, and sets up the
Python path, similarly to the behavior of the function it replaces. The
following shows the part that buildout inserts.
>>> sys.stdout.write('#\n'); cat(site_path)
... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
#...
def addsitepackages(known_paths):
paths = []
paths[0:0] = [ # eggs
'/interpreter/eggs/demo-0.3-pyN.N.egg',
'/interpreter/eggs/demoneeded-1.1-pyN.N.egg'
]
# Process all dirs. Look for .pth files. If they exist, defer
# processing "import" varieties.
dotpth = os.extsep + "pth"
deferred = []
for path in reversed(paths):
# Duplicating addsitedir.
sitedir, sitedircase = makepath(path)
if not sitedircase in known_paths and os.path.exists(sitedir):
sys.path.insert(0, sitedir)
known_paths.add(sitedircase)
try:
names = os.listdir(sitedir)
except os.error:
continue
names = [name for name in names if name.endswith(dotpth)]
names.sort()
for name in names:
# Duplicating addpackage.
fullname = os.path.join(sitedir, name)
try:
f = open(fullname, "rU")
except IOError:
continue
try:
for line in f:
if line.startswith("#"):
continue
if (line.startswith("import ") or
line.startswith("import ")):
# This line is supposed to be executed. It
# might be a setuptools namespace package
# installed with a system package manager.
# Defer this so we can process egg namespace
# packages first, or else the eggs with the same
# namespace will be ignored.
deferred.append((sitedir, name, fullname, line))
continue
line = line.rstrip()
dir, dircase = makepath(sitedir, line)
if not dircase in known_paths and os.path.exists(dir):
sys.path.append(dir)
known_paths.add(dircase)
finally:
f.close()
# Process "import ..." .pth lines.
for sitedir, name, fullname, line in deferred:
# Note that some lines--such as the one setuptools writes for
# namespace packages--expect some or all of sitedir, name, and
# fullname to be present in the frame locals, as it is in
# ``addpackage``.
try:
exec line
except:
print "Error in %s" % (fullname,)
raise
global addsitepackages
addsitepackages = original_addsitepackages
return known_paths
<BLANKLINE>
buildout_addsitepackages = addsitepackages
<BLANKLINE>
def original_addsitepackages(known_paths):...
As you can see, it manipulates the path to insert the eggs and then processes
any .pth files. The lines in the .pth files that use the "import" feature
are deferred because it is a pattern we will need in a later example, when we
show how we can add site packages, and handle competing namespace packages
in both site packages and eggs.
Here are some examples of the interpreter in use.
>>> print call_py(interpreter_path, "print 16+26")
42
<BLANKLINE>
>>> res = call_py(interpreter_path, "import sys; print sys.path")
>>> print res # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
['',
'/interpreter/eggs/demo-0.3-pyN.N.egg',
'/interpreter/eggs/demoneeded-1.1-pyN.N.egg',
'/interpreter/parts/interpreter',
...]
<BLANKLINE>
>>> clean_paths = eval(res.strip()) # This is used later for comparison.
If you provide initialization, it goes in sitecustomize.py.
>>> def reset_interpreter():
... # This is necessary because, in our tests, the timestamps of the
... # .pyc files are not outdated when we want them to be.
... rmdir(interpreter_bin_dir)
... mkdir(interpreter_bin_dir)
... rmdir(interpreter_parts_dir)
... mkdir(interpreter_parts_dir)
...
>>> reset_interpreter()
>>> initialization_string = """\
... import os
... os.environ['FOO'] = 'bar baz bing shazam'"""
>>> generated = zc.buildout.easy_install.generate_scripts(
... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
... interpreter='py', initialization=initialization_string)
>>> cat(sitecustomize_path)
import os
os.environ['FOO'] = 'bar baz bing shazam'
>>> print call_py(interpreter_path, "import os; print os.environ['FOO']")
bar baz bing shazam
<BLANKLINE>
If you use relative paths, this affects the interpreter and site.py.
>>> reset_interpreter()
>>> generated = zc.buildout.easy_install.generate_scripts(
... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
... interpreter='py', relative_paths=interpreter_dir)
>>> cat(py_path)
#!/usr/bin/python2.4 -S
<BLANKLINE>
import os
<BLANKLINE>
join = os.path.join
base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
base = os.path.dirname(base)
<BLANKLINE>
import os
import sys
<BLANKLINE>
argv = [sys.executable] + sys.argv[1:]
environ = os.environ.copy()
path = join(base, 'parts/interpreter')
if environ.get('PYTHONPATH'):
path = os.pathsep.join([path, environ['PYTHONPATH']])
environ['PYTHONPATH'] = path
os.execve(sys.executable, argv, environ)
For site.py, we again show only the pertinent parts. Notice that the egg
paths join a base to a path, as with the use of this argument in the
``scripts`` function.
>>> sys.stdout.write('#\n'); cat(site_path) # doctest: +ELLIPSIS
#...
def addsitepackages(known_paths):
paths = []
<BLANKLINE>
import os
<BLANKLINE>
join = os.path.join
base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
base = os.path.dirname(base)
base = os.path.dirname(base)
paths[0:0] = [ # eggs
join(base, 'eggs/demo-0.3-pyN.N.egg'),
join(base, 'eggs/demoneeded-1.1-pyN.N.egg')
]...
The paths resolve in practice as you would expect.
>>> print call_py(interpreter_path,
... "import sys, pprint; pprint.pprint(sys.path)")
... # doctest: +ELLIPSIS
['',
'/interpreter/eggs/demo-0.3-py2.4.egg',
'/interpreter/eggs/demoneeded-1.1-py2.4.egg',
'/interpreter/parts/interpreter',
...]
<BLANKLINE>
The ``extra_paths`` argument affects the path in site.py. Notice that
/interpreter/other is added after the eggs.
>>> reset_interpreter()
>>> mkdir(interpreter_dir, 'other')
>>> generated = zc.buildout.easy_install.generate_scripts(
... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
... interpreter='py', extra_paths=[join(interpreter_dir, 'other')])
>>> sys.stdout.write('#\n'); cat(site_path) # doctest: +ELLIPSIS
#...
def addsitepackages(known_paths):
paths = []
paths[0:0] = [ # eggs
'/interpreter/eggs/demo-0.3-pyN.N.egg',
'/interpreter/eggs/demoneeded-1.1-pyN.N.egg',
'/interpreter/other'
]...
>>> print call_py(interpreter_path,
... "import sys, pprint; pprint.pprint(sys.path)")
... # doctest: +ELLIPSIS
['',
'/interpreter/eggs/demo-0.3-pyN.N.egg',
'/interpreter/eggs/demoneeded-1.1-pyN.N.egg',
'/interpreter/other',
'/interpreter/parts/interpreter',
...]
<BLANKLINE>
The ``generate_scripts`` function: using site-packages
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The ``generate_scripts`` function supports including site packages. This has
some advantages and some serious dangers.
A typical reason to include site-packages is that it is easier to
install one or more dependencies in your Python than it is with
buildbot. Some packages, such as lxml or Python PostgreSQL integration,
have dependencies that can be much easier to build and/or install using
other mechanisms, such as your operating system's package manager. By
installing some core packages into your Python's site-packages, this can
significantly simplify some application installations.
However, doing this has a significant danger. One of the primary goals
of buildout is to provide repeatability. Some packages (one of the
better known Python openid packages, for instance) change their behavior
depending on what packages are available. If Python curl bindings are
available, these may be preferred by the library. If a certain XML
package is installed, it may be preferred by the library. These hidden
choices may cause small or large behavior differences. The fact that
they can be rarely encountered can actually make it worse: you forget
that this might be a problem, and debugging the differences can be
difficult. If you allow site-packages to be included in your buildout,
and the Python you use is not managed precisely by your application (for
instance, it is a system Python), you open yourself up to these
possibilities. Don't be unaware of the dangers.
That explained, let's see how it works. Unfortunately, because of how
setuptools namespace packages are implemented differently for operating
system packages (debs or rpms) and normal installation, there's a tricky
dance.
>>> reset_interpreter()
>>> generated = zc.buildout.easy_install.generate_scripts(
... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
... interpreter='py', add_site_packages=True)
>>> sys.stdout.write('#\n'); cat(site_path)
... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
#...
def addsitepackages(known_paths):
paths = [ # These are the underlying Python's site-packages.
'...']
sys.path[0:0] = paths
known_paths.update([os.path.normcase(os.path.abspath(p)) for p in paths])
try:
import pkg_resources
except ImportError:
# No namespace packages in sys.path; no fixup needed.
pkg_resources = None
paths[0:0] = [ # eggs
'/interpreter/eggs/demo-0.3-pyN.N.egg',
'/interpreter/eggs/demoneeded-1.1-pyN.N.egg'
]
# Process all dirs. Look for .pth files. If they exist, defer
# processing "import" varieties.
dotpth = os.extsep + "pth"
deferred = []
for path in reversed(paths):
# Duplicating addsitedir.
sitedir, sitedircase = makepath(path)
if not sitedircase in known_paths and os.path.exists(sitedir):
sys.path.insert(0, sitedir)
known_paths.add(sitedircase)
try:
names = os.listdir(sitedir)
except os.error:
continue
names = [name for name in names if name.endswith(dotpth)]
names.sort()
for name in names:
# Duplicating addpackage.
fullname = os.path.join(sitedir, name)
try:
f = open(fullname, "rU")
except IOError:
continue
try:
for line in f:
if line.startswith("#"):
continue
if (line.startswith("import ") or
line.startswith("import ")):
# This line is supposed to be executed. It
# might be a setuptools namespace package
# installed with a system package manager.
# Defer this so we can process egg namespace
# packages first, or else the eggs with the same
# namespace will be ignored.
deferred.append((sitedir, name, fullname, line))
continue
line = line.rstrip()
dir, dircase = makepath(sitedir, line)
if not dircase in known_paths and os.path.exists(dir):
sys.path.append(dir)
known_paths.add(dircase)
finally:
f.close()
if pkg_resources is not None:
# There may be namespace packages in sys.path. This is much faster
# than importing pkg_resources after the sys.path has a large number
# of eggs.
for p in sys.path:
pkg_resources.fixup_namespace_packages(p)
# Process "import ..." .pth lines.
for sitedir, name, fullname, line in deferred:
# Note that some lines--such as the one setuptools writes for
# namespace packages--expect some or all of sitedir, name, and
# fullname to be present in the frame locals, as it is in
# ``addpackage``.
try:
exec line
except:
print "Error in %s" % (fullname,)
raise
global addsitepackages
addsitepackages = original_addsitepackages
return known_paths
<BLANKLINE>
buildout_addsitepackages = addsitepackages
<BLANKLINE>
def original_addsitepackages(known_paths):...
As you can see, the script now first tries to import pkg_resources. If it
exists, then we need to process egg files specially to look for namespace
packages there *before* we process process lines in .pth files that use the
"import" feature--lines that might be part of the setuptools namespace
package implementation for system packages, as mentioned above, and that
must come after processing egg namespaces.
Here's an example of the new script in use. Other documents and tests in
this package give the feature a more thorough workout, but this should
give you an idea of the feature.
>>> res = call_py(interpreter_path, "import sys; print sys.path")
>>> print res # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
['',
'/interpreter/eggs/demo-0.3-py2.4.egg',
'/interpreter/eggs/demoneeded-1.1-py2.4.egg',
'...',
'/interpreter/parts/interpreter',
...]
<BLANKLINE>
The clean_paths gathered earlier is a subset of this full list of paths.
>>> full_paths = eval(res.strip())
>>> len(clean_paths) < len(full_paths)
True
>>> set(os.path.normpath(p) for p in clean_paths).issubset(
... os.path.normpath(p) for p in full_paths)
True
The ``exec_sitecustomize`` argument does the same thing for the
sitecustomize module--it allows you to include the code from the
sitecustomize module in the underlying Python if you set the argument to
True. The z3c.recipe.scripts package sets up the full environment necessary
to demonstrate this piece.
The ``generate_scripts`` function: writing scripts for entry points
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
All of the examples so far for this function have been creating
interpreters. The function can also write scripts for entry
points. They are almost identical to the scripts that we saw for the
``scripts`` function except that they ``import site`` after setting the
sys.path to include our custom site.py and sitecustomize.py files. These
files then initialize the Python environment as we have already seen. Let's
see a simple example.
>>> reset_interpreter()
>>> generated = zc.buildout.easy_install.generate_scripts(
... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
... reqs=['demo'])
As before, in Windows, 2 files are generated for each script. A script
file, ending in '-script.py', and an exe file that allows the script
to be invoked directly without having to specify the Python
interpreter and without having to provide a '.py' suffix. This is in addition
to the site.py and sitecustomize.py files that are generated as with our
interpreter examples above.
>>> if sys.platform == 'win32':
... demo_path = os.path.join(interpreter_bin_dir, 'demo-script.py')
... expected = [sitecustomize_path,
... site_path,
... os.path.join(interpreter_bin_dir, 'demo.exe'),
... demo_path]
... else:
... demo_path = os.path.join(interpreter_bin_dir, 'demo')
... expected = [sitecustomize_path, site_path, demo_path]
...
>>> assert generated == expected, repr((generated, expected))
The demo script runs the entry point defined in the demo egg:
>>> cat(demo_path) # doctest: +NORMALIZE_WHITESPACE
#!/usr/local/bin/python2.4 -S
<BLANKLINE>
import sys
sys.path[0:0] = [
'/interpreter/parts/interpreter',
]
<BLANKLINE>
<BLANKLINE>
import site # imports custom buildbot-generated site.py
<BLANKLINE>
import eggrecipedemo
<BLANKLINE>
if __name__ == '__main__':
eggrecipedemo.main()
>>> demo_call = join(interpreter_bin_dir, 'demo')
>>> if sys.platform == 'win32':
... demo_call = '"%s"' % demo_call
>>> print system(demo_call)
3 1
<BLANKLINE>
There are a few differences from the ``scripts`` function. First, the
``reqs`` argument (an iterable of string requirements or entry point
tuples) is a keyword argument here. We see that in the example above.
Second, the ``arguments`` argument is now named ``script_arguments`` to
try and clarify that it does not affect interpreters. While the
``initialization`` argument continues to affect both the interpreters
and the entry point scripts, if you have initialization that is only
pertinent to the entry point scripts, you can use the
``script_initialization`` argument.
Let's see ``script_arguments`` and ``script_initialization`` in action.
>>> reset_interpreter()
>>> generated = zc.buildout.easy_install.generate_scripts(
... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
... reqs=['demo'], script_arguments='1, 2',
... script_initialization='import os\nos.chdir("foo")')
>>> cat(demo_path) # doctest: +NORMALIZE_WHITESPACE
#!/usr/local/bin/python2.4 -S
import sys
sys.path[0:0] = [
'/interpreter/parts/interpreter',
]
<BLANKLINE>
import site # imports custom buildbot-generated site.py
import os
os.chdir("foo")
<BLANKLINE>
import eggrecipedemo
<BLANKLINE>
if __name__ == '__main__':
eggrecipedemo.main(1, 2)
Handling custom build options for extensions provided in source distributions
-----------------------------------------------------------------------------
...
...
@@ -1768,7 +1208,6 @@ Now, if we install demo, and extdemo:
GET 200 /demo-0.2-py2.4.egg
GET 404 /index/demoneeded/
GET 200 /demoneeded-1.1.zip
GET 404 /index/setuptools/
>>> zc.buildout.easy_install.build(
... 'extdemo', dest,
...
...
@@ -1811,7 +1250,6 @@ Now when we install the distributions:
GET 404 /index/demo/
GET 200 /index/
GET 404 /index/demoneeded/
GET 404 /index/setuptools/
>>> zc.buildout.easy_install.build(
... 'extdemo', dest,
...
...
src/zc/buildout/testing.py
View file @
6107db1f
...
...
@@ -28,7 +28,6 @@ import socket
import
subprocess
import
sys
import
tempfile
import
textwrap
import
threading
import
time
import
urllib2
...
...
@@ -106,16 +105,6 @@ def system(command, input=''):
e
.
close
()
return
result
def
call_py
(
interpreter
,
cmd
,
flags
=
None
):
if
sys
.
platform
==
'win32'
:
args
=
[
'"%s"'
%
arg
for
arg
in
(
interpreter
,
flags
,
cmd
)
if
arg
]
args
.
insert
(
-
1
,
'"-c"'
)
return
system
(
'"%s"'
%
' '
.
join
(
args
))
else
:
cmd
=
repr
(
cmd
)
return
system
(
' '
.
join
(
arg
for
arg
in
(
interpreter
,
flags
,
'-c'
,
cmd
)
if
arg
))
def
get
(
url
):
return
urllib2
.
urlopen
(
url
).
read
()
...
...
@@ -127,12 +116,7 @@ def _runsetup(setup, executable, *args):
args
=
[
zc
.
buildout
.
easy_install
.
_safe_arg
(
arg
)
for
arg
in
args
]
args
.
insert
(
0
,
'-q'
)
env
=
dict
(
os
.
environ
)
if
executable
==
sys
.
executable
:
env
[
'PYTHONPATH'
]
=
setuptools_location
# else pass an executable that has setuptools! See testselectingpython.py.
args
.
append
(
env
)
args
.
append
(
dict
(
os
.
environ
,
PYTHONPATH
=
setuptools_location
))
here
=
os
.
getcwd
()
try
:
...
...
@@ -151,11 +135,6 @@ def sdist(setup, dest):
def
bdist_egg
(
setup
,
executable
,
dest
):
_runsetup
(
setup
,
executable
,
'bdist_egg'
,
'-d'
,
dest
)
def
sys_install
(
setup
,
dest
):
_runsetup
(
setup
,
sys
.
executable
,
'install'
,
'--install-purelib'
,
dest
,
'--record'
,
os
.
path
.
join
(
dest
,
'__added_files__'
),
'--single-version-externally-managed'
)
def
find_python
(
version
):
e
=
os
.
environ
.
get
(
'PYTHON%s'
%
version
)
if
e
is
not
None
:
...
...
@@ -223,24 +202,6 @@ def wait_until(label, func, *args, **kw):
time
.
sleep
(
0.01
)
raise
ValueError
(
'Timed out waiting for: '
+
label
)
def
make_buildout
():
# Create a basic buildout.cfg to avoid a warning from buildout:
open
(
'buildout.cfg'
,
'w'
).
write
(
"[buildout]
\
n
parts =
\
n
"
)
# Use the buildout bootstrap command to create a buildout
zc
.
buildout
.
buildout
.
Buildout
(
'buildout.cfg'
,
[(
'buildout'
,
'log-level'
,
'WARNING'
),
# trick bootstrap into putting the buildout develop egg
# in the eggs dir.
(
'buildout'
,
'develop-eggs-directory'
,
'eggs'
),
]
).
bootstrap
([])
# Create the develop-eggs dir, which didn't get created the usual
# way due to the trick above:
os
.
mkdir
(
'develop-eggs'
)
def
buildoutSetUp
(
test
):
test
.
globs
[
'__tear_downs'
]
=
__tear_downs
=
[]
...
...
@@ -294,7 +255,27 @@ def buildoutSetUp(test):
sample
=
tmpdir
(
'sample-buildout'
)
os
.
chdir
(
sample
)
make_buildout
()
# Create a basic buildout.cfg to avoid a warning from buildout:
open
(
'buildout.cfg'
,
'w'
).
write
(
"[buildout]
\
n
parts =
\
n
"
)
# Use the buildout bootstrap command to create a buildout
zc
.
buildout
.
buildout
.
Buildout
(
'buildout.cfg'
,
[(
'buildout'
,
'log-level'
,
'WARNING'
),
# trick bootstrap into putting the buildout develop egg
# in the eggs dir.
(
'buildout'
,
'develop-eggs-directory'
,
'eggs'
),
]
).
bootstrap
([])
# Create the develop-eggs dir, which didn't get created the usual
# way due to the trick above:
os
.
mkdir
(
'develop-eggs'
)
def
start_server
(
path
):
port
,
thread
=
_start_server
(
path
,
name
=
path
)
...
...
@@ -302,40 +283,6 @@ def buildoutSetUp(test):
register_teardown
(
lambda
:
stop_server
(
url
,
thread
))
return
url
def
make_py
(
initialization
=
''
):
"""Returns paths to new executable and to its site-packages.
"""
buildout
=
tmpdir
(
'executable_buildout'
)
site_packages_dir
=
os
.
path
.
join
(
buildout
,
'site-packages'
)
mkdir
(
site_packages_dir
)
old_wd
=
os
.
getcwd
()
os
.
chdir
(
buildout
)
make_buildout
()
initialization
=
'
\
n
'
.
join
(
' '
+
line
for
line
in
initialization
.
split
(
'
\
n
'
))
install_develop
(
'zc.recipe.egg'
,
os
.
path
.
join
(
buildout
,
'develop-eggs'
))
install_develop
(
'z3c.recipe.scripts'
,
os
.
path
.
join
(
buildout
,
'develop-eggs'
))
write
(
'buildout.cfg'
,
textwrap
.
dedent
(
'''
\
[buildout]
parts = py
[py]
recipe = z3c.recipe.scripts
interpreter = py
initialization =
%(initialization)s
extra-paths = %(site-packages)s
eggs = setuptools
'''
)
%
{
'initialization'
:
initialization
,
'site-packages'
:
site_packages_dir
})
system
(
os
.
path
.
join
(
buildout
,
'bin'
,
'buildout'
))
os
.
chdir
(
old_wd
)
return
(
os
.
path
.
join
(
buildout
,
'bin'
,
'py'
),
site_packages_dir
)
test
.
globs
.
update
(
dict
(
sample_buildout
=
sample
,
ls
=
ls
,
...
...
@@ -346,7 +293,6 @@ def buildoutSetUp(test):
tmpdir
=
tmpdir
,
write
=
write
,
system
=
system
,
call_py
=
call_py
,
get
=
get
,
cd
=
(
lambda
*
path
:
os
.
chdir
(
os
.
path
.
join
(
*
path
))),
join
=
os
.
path
.
join
,
...
...
@@ -355,7 +301,6 @@ def buildoutSetUp(test):
start_server
=
start_server
,
buildout
=
os
.
path
.
join
(
sample
,
'bin'
,
'buildout'
),
wait_until
=
wait_until
,
make_py
=
make_py
))
zc
.
buildout
.
easy_install
.
prefer_final
(
prefer_final
)
...
...
@@ -547,14 +492,10 @@ def _normalize_path(match):
path
=
path
[
1
:]
return
'/'
+
path
.
replace
(
os
.
path
.
sep
,
'/'
)
if
sys
.
platform
==
'win32'
:
sep
=
r'[\\/]'
# Windows uses both sometimes.
else
:
sep
=
re
.
escape
(
os
.
path
.
sep
)
normalize_path
=
(
re
.
compile
(
r'''[^'" \t\n\r]+
%(sep)s_[Tt][Ee][Ss][Tt]_
%(sep)s([^"' \t\n\r]+)'''
%
dict
(
sep
=
sep
)),
r'''[^'" \t\n\r]+
\
%(sep)s_[T
t][Ee][Ss][Tt]_\
%(sep)s([^
"' \t\n\r]+)'''
%
dict
(
sep
=
os
.
path
.
sep
)),
_normalize_path
,
)
...
...
src/zc/buildout/tests.py
View file @
6107db1f
...
...
@@ -53,7 +53,6 @@ We should be able to deal with setup scripts that aren't setuptools based.
>>> ls('develop-eggs')
- foo.egg-link
- z3c.recipe.scripts.egg-link
- zc.recipe.egg.egg-link
"""
...
...
@@ -85,7 +84,6 @@ We should be able to deal with setup scripts that aren't setuptools based.
>>> ls('develop-eggs')
- foo.egg-link
- z3c.recipe.scripts.egg-link
- zc.recipe.egg.egg-link
>>> print system(join('bin', 'buildout')+' -vvv'), # doctest: +ELLIPSIS
...
...
@@ -670,7 +668,6 @@ Create a develop egg:
>>> ls('develop-eggs')
- foox.egg-link
- z3c.recipe.scripts.egg-link
- zc.recipe.egg.egg-link
Create another:
...
...
@@ -695,7 +692,6 @@ Create another:
>>> ls('develop-eggs')
- foox.egg-link
- fooy.egg-link
- z3c.recipe.scripts.egg-link
- zc.recipe.egg.egg-link
Remove one:
...
...
@@ -713,7 +709,6 @@ It is gone
>>> ls('develop-eggs')
- fooy.egg-link
- z3c.recipe.scripts.egg-link
- zc.recipe.egg.egg-link
Remove the other:
...
...
@@ -728,7 +723,6 @@ Remove the other:
All gone
>>> ls('develop-eggs')
- z3c.recipe.scripts.egg-link
- zc.recipe.egg.egg-link
'''
...
...
@@ -803,7 +797,6 @@ On the other hand, if we have a regular egg, rather than a develop egg:
... + join(sample_buildout, 'eggs'))
>>> ls('develop-eggs')
- z3c.recipe.scripts.egg-link
- zc.recipe.egg.egg-link
>>> ls('eggs') # doctest: +ELLIPSIS
...
...
@@ -1776,233 +1769,6 @@ def bug_105081_Specific_egg_versions_are_ignored_when_newer_eggs_are_around():
1 2
"""
def
versions_section_ignored_for_dependency_in_favor_of_site_packages
():
r"""
This is a test for a bugfix.
The error showed itself when at least two dependencies were in a shared
location like site-packages, and the first one met the "versions" setting. The
first dependency would be added, but subsequent dependencies from the same
location (e.g., site-packages) would use the version of the package found in
the shared location, ignoring the version setting.
We begin with a Python that has demoneeded version 1.1 installed and a
demo version 0.3, all in a site-packages-like shared directory. We need
to create this. ``eggrecipedemo.main()`` shows the number after the dot
(that is, ``X`` in ``1.X``), for the demo package and the demoneeded
package, so this demonstrates that our Python does in fact have demo
version 0.3 and demoneeded version 1.1.
>>> py_path = make_py_with_system_install(make_py, sample_eggs)
>>> print call_py(
... py_path,
... "import tellmy.version; print tellmy.version.__version__"),
1.1
Now here's a setup that would expose the bug, using the
zc.buildout.easy_install API.
>>> example_dest = tmpdir('example_dest')
>>> workingset = zc.buildout.easy_install.install(
... ['tellmy.version'], example_dest, links=[sample_eggs],
... executable=py_path,
... index=None,
... versions={'tellmy.version': '1.0'})
>>> for dist in workingset:
... res = str(dist)
... if res.startswith('tellmy.version'):
... print res
... break
tellmy.version 1.0
Before the bugfix, the desired tellmy.version distribution would have
been blocked the one in site-packages.
"""
def
handle_namespace_package_in_both_site_packages_and_buildout_eggs
():
r"""
If you have the same namespace package in both site-packages and in
buildout, we need to be very careful that faux-Python-executables and
scripts generated by easy_install.generate_scripts correctly combine the two.
We show this with the local recipe that uses the function, z3c.recipe.scripts.
To demonstrate this, we will create three packages: tellmy.version 1.0,
tellmy.version 1.1, and tellmy.fortune 1.0. tellmy.version 1.1 is installed.
>>> py_path = make_py_with_system_install(make_py, sample_eggs)
>>> print call_py(
... py_path,
... "import tellmy.version; print tellmy.version.__version__")
1.1
<BLANKLINE>
Now we will create a buildout that creates a script and a faux-Python script.
We want to see that both can successfully import the specified versions of
tellmy.version and tellmy.fortune.
>>> write('buildout.cfg',
... '''
... [buildout]
... parts = eggs
... find-links = %(link_server)s
...
... [primed_python]
... executable = %(py_path)s
...
... [eggs]
... recipe = z3c.recipe.scripts
... python = primed_python
... interpreter = py
... add-site-packages = true
... eggs = tellmy.version == 1.0
... tellmy.fortune == 1.0
... demo
... script-initialization =
... import tellmy.version
... print tellmy.version.__version__
... import tellmy.fortune
... print tellmy.fortune.__version__
... ''' % globals())
>>> print system(buildout)
Installing eggs.
Getting distribution for 'tellmy.version==1.0'.
Got tellmy.version 1.0.
Getting distribution for 'tellmy.fortune==1.0'.
Got tellmy.fortune 1.0.
Getting distribution for 'demo'.
Got demo 0.4c1.
Getting distribution for 'demoneeded'.
Got demoneeded 1.2c1.
Generated script '/sample-buildout/bin/demo'.
Generated interpreter '/sample-buildout/bin/py'.
<BLANKLINE>
Finally, we are ready for the actual test. Prior to the bug fix that
this tests, the results of both calls below was the following::
1.1
Traceback (most recent call last):
...
ImportError: No module named fortune
<BLANKLINE>
In other words, we got the site-packages version of tellmy.version, and
we could not import tellmy.fortune at all. The following are the correct
results for the interpreter and for the script.
>>> print call_py(
... join('bin', 'py'),
... "import tellmy.version; " +
... "print tellmy.version.__version__; " +
... "import tellmy.fortune; " +
... "print tellmy.fortune.__version__") # doctest: +ELLIPSIS
1.0
1.0...
>>> print system(join('bin', 'demo'))
1.0
1.0
4 2
<BLANKLINE>
"""
def
handle_sys_path_version_hack
():
r"""
This is a test for a bugfix.
If you use a Python that has a different version of one of your
dependencies, and the new package tries to do sys.path tricks in the
setup.py to get a __version__, and it uses namespace packages, the older
package will be loaded first, making the setup version the wrong number.
While very arguably packages simply shouldn't do this, some do, and we
don't want buildout to fall over when they do.
To demonstrate this, we will need to create a distribution that has one of
these unpleasant tricks, and a Python that has an older version installed.
>>> py_path, site_packages_path = make_py()
>>> for version in ('1.0', '1.1'):
... tmp = tempfile.mkdtemp()
... try:
... write(tmp, 'README.txt', '')
... mkdir(tmp, 'src')
... mkdir(tmp, 'src', 'tellmy')
... write(tmp, 'src', 'tellmy', '__init__.py',
... "__import__("
... "'pkg_resources').declare_namespace(__name__)\n")
... mkdir(tmp, 'src', 'tellmy', 'version')
... write(tmp, 'src', 'tellmy', 'version',
... '__init__.py', '__version__=%r\n' % version)
... write(
... tmp, 'setup.py',
... "from setuptools import setup\n"
... "import sys\n"
... "sys.path.insert(0, 'src')\n"
... "from tellmy.version import __version__\n"
... "setup(\n"
... " name='tellmy.version',\n"
... " package_dir = {'': 'src'},\n"
... " packages = ['tellmy', 'tellmy.version'],\n"
... " install_requires = ['setuptools'],\n"
... " namespace_packages=['tellmy'],\n"
... " zip_safe=True, version=__version__,\n"
... " author='bob', url='bob', author_email='bob')\n"
... )
... zc.buildout.testing.sdist(tmp, sample_eggs)
... if version == '1.0':
... # We install the 1.0 version in site packages the way a
... # system packaging system (debs, rpms) would do it.
... zc.buildout.testing.sys_install(tmp, site_packages_path)
... finally:
... shutil.rmtree(tmp)
>>> print call_py(
... py_path,
... "import tellmy.version; print tellmy.version.__version__")
1.0
<BLANKLINE>
>>> write('buildout.cfg',
... '''
... [buildout]
... parts = eggs
... find-links = %(sample_eggs)s
...
... [primed_python]
... executable = %(py_path)s
...
... [eggs]
... recipe = zc.recipe.egg:eggs
... python = primed_python
... eggs = tellmy.version == 1.1
... ''' % globals())
Before the bugfix, running this buildout would generate this error:
Installing eggs.
Getting distribution for 'tellmy.version==1.1'.
Installing tellmy.version 1.1
Caused installation of a distribution:
tellmy.version 1.0
with a different version.
Got None.
While:
Installing eggs.
Error: There is a version conflict.
We already have: tellmy.version 1.0
<BLANKLINE>
The bugfix was simply to add Python's "-S" option when calling
easyinstall (see zc.buildout.easy_install.Installer._call_easy_install).
Now the install works correctly, as seen here.
>>> print system(buildout)
Installing eggs.
Getting distribution for 'tellmy.version==1.1'.
Got tellmy.version 1.1.
<BLANKLINE>
"""
if
sys
.
version_info
>
(
2
,
4
):
def
test_exit_codes
():
"""
...
...
@@ -2601,7 +2367,6 @@ Distribution setup scripts can import modules in the distribution directory:
>>> ls('develop-eggs')
- foo.egg-link
- z3c.recipe.scripts.egg-link
- zc.recipe.egg.egg-link
"""
...
...
@@ -2889,44 +2654,6 @@ def increment_on_command_line():
######################################################################
def
make_py_with_system_install
(
make_py
,
sample_eggs
):
from
zc.buildout.testing
import
write
,
mkdir
py_path
,
site_packages_path
=
make_py
()
for
pkg
,
version
in
((
'version'
,
'1.0'
),
(
'version'
,
'1.1'
),
(
'fortune'
,
'1.0'
)):
tmp
=
tempfile
.
mkdtemp
()
try
:
write
(
tmp
,
'README.txt'
,
''
)
mkdir
(
tmp
,
'src'
)
mkdir
(
tmp
,
'src'
,
'tellmy'
)
write
(
tmp
,
'src'
,
'tellmy'
,
'__init__.py'
,
"__import__("
"'pkg_resources').declare_namespace(__name__)
\
n
"
)
mkdir
(
tmp
,
'src'
,
'tellmy'
,
pkg
)
write
(
tmp
,
'src'
,
'tellmy'
,
pkg
,
'__init__.py'
,
'__version__=%r
\
n
'
%
version
)
write
(
tmp
,
'setup.py'
,
"from setuptools import setup
\
n
"
"setup(
\
n
"
" name='tellmy.%(pkg)s',
\
n
"
" package_dir = {'': 'src'},
\
n
"
" packages = ['tellmy', 'tellmy.%(pkg)s'],
\
n
"
" install_requires = ['setuptools'],
\
n
"
" namespace_packages=['tellmy'],
\
n
"
" zip_safe=True, version=%(version)r,
\
n
"
" author='bob', url='bob', author_email='bob')
\
n
"
%
locals
()
)
zc
.
buildout
.
testing
.
sdist
(
tmp
,
sample_eggs
)
if
pkg
==
'version'
and
version
==
'1.1'
:
# We install the 1.1 version in site packages the way a
# system packaging system (debs, rpms) would do it.
zc
.
buildout
.
testing
.
sys_install
(
tmp
,
site_packages_path
)
finally
:
shutil
.
rmtree
(
tmp
)
return
py_path
def
create_sample_eggs
(
test
,
executable
=
sys
.
executable
):
write
=
test
.
globs
[
'write'
]
dest
=
test
.
globs
[
'sample_eggs'
]
...
...
@@ -3049,7 +2776,6 @@ def easy_install_SetUp(test):
test
.
globs
[
'sample_eggs'
])
test
.
globs
[
'update_extdemo'
]
=
lambda
:
add_source_dist
(
test
,
1.5
)
zc
.
buildout
.
testing
.
install_develop
(
'zc.recipe.egg'
,
test
)
zc
.
buildout
.
testing
.
install_develop
(
'z3c.recipe.scripts'
,
test
)
egg_parse
=
re
.
compile
(
'([0-9a-zA-Z_.]+)-([0-9a-zA-Z_.]+)-py(
\
d[.]
\
d).egg$'
).
match
...
...
@@ -3154,7 +2880,7 @@ def test_suite():
'
We
have
a
develop
egg
:
zc
.
buildout
X
.
X
.
'),
(re.compile(r'
\\
[
\\
]
?
'), '
/
'),
(re.compile('
WindowsError
'), '
OSError
'),
(re.compile(r'
\
[
Error
\
d
+
\
]
Cannot
create
a
file
'
(re.compile(r'
\
[
Error
17
\
]
Cannot
create
a
file
'
r'
when
that
file
already
exists
:
'),
'
[
Errno
17
]
File
exists
:
'
),
...
...
@@ -3195,7 +2921,7 @@ def test_suite():
doctest.DocFileSuite(
'
easy_install
.
txt
', '
downloadcache
.
txt
', '
dependencylinks
.
txt
',
'
allowhosts
.
txt
', '
unzip
.
txt
',
'
allowhosts
.
txt
', '
unzip
.
txt
',
'
upgrading_distribute
.
txt
',
setUp=easy_install_SetUp,
tearDown=zc.buildout.testing.buildoutTearDown,
checker=renormalizing.RENormalizing([
...
...
@@ -3207,11 +2933,6 @@ def test_suite():
(re.compile('
extdemo
[.]
pyd
'), '
extdemo
.
so
'),
(re.compile('
[
-
d
]
setuptools
-
\
S
+
[.]
egg
'), '
setuptools
.
egg
'),
(re.compile(r'
\\
[
\\
]
?
'), '
/
'),
(re.compile(r'
\
#!\S+\bpython\S*'), '#!/usr/bin/python'),
# Normalize generate_script's Windows interpreter to UNIX:
(
re
.
compile
(
r'\nimport subprocess\n'
),
'
\
n
'
),
(
re
.
compile
(
'subprocess
\
\
.call
\
\
(argv, env=environ
\
\
)'
),
'os.execve(sys.executable, argv, environ)'
),
]+(sys.version_info < (2, 5) and [
(re.compile('
.
*
No
module
named
runpy
.
*
', re.S), ''),
(re.compile('
.
*
usage
:
pdb
.
py
scriptfile
.
*
', re.S), ''),
...
...
src/zc/buildout/testselectingpython.py
View file @
6107db1f
...
...
@@ -11,7 +11,7 @@
# FOR A PARTICULAR PURPOSE.
#
##############################################################################
import
os
,
re
,
s
ubprocess
,
sys
,
textwrap
,
unittest
import
os
,
re
,
s
ys
,
unittest
from
zope.testing
import
doctest
,
renormalizing
import
zc.buildout.tests
import
zc.buildout.testing
...
...
@@ -36,40 +36,12 @@ We can specify a specific Python executable.
>>> ls(dest)
d demo-0.3-py%(other_version)s.egg
d demoneeded-1.1-py%(other_version)s.egg
d setuptools-0.6-py%(other_version)s.egg
"""
%
dict
(
other_version
=
other_version
)
)
def
multi_python
(
test
):
other_executable
=
zc
.
buildout
.
testing
.
find_python
(
other_version
)
command
=
textwrap
.
dedent
(
'''
\
try:
import setuptools
except ImportError:
import sys
sys.exit(1)
'''
)
if
subprocess
.
call
([
other_executable
,
'-c'
,
command
],
env
=
os
.
environ
):
# the other executable does not have setuptools. Get setuptools.
# We will do this using the same tools we are testing, for better or
# worse. Alternatively, we could try using bootstrap.
executable_dir
=
test
.
globs
[
'tmpdir'
](
'executable_dir'
)
executable_parts
=
os
.
path
.
join
(
executable_dir
,
'parts'
)
test
.
globs
[
'mkdir'
](
executable_parts
)
ws
=
zc
.
buildout
.
easy_install
.
install
(
[
'setuptools'
],
executable_dir
,
index
=
'http://www.python.org/pypi/'
,
always_unzip
=
True
,
executable
=
other_executable
)
zc
.
buildout
.
easy_install
.
generate_scripts
(
executable_dir
,
ws
,
other_executable
,
executable_parts
,
reqs
=
[
'setuptools'
],
interpreter
=
'py'
)
original_executable
=
other_executable
other_executable
=
os
.
path
.
join
(
executable_dir
,
'py'
)
assert
not
subprocess
.
call
(
[
other_executable
,
'-c'
,
command
],
env
=
os
.
environ
),
(
'test set up failed'
)
sample_eggs
=
test
.
globs
[
'tmpdir'
](
'sample_eggs'
)
os
.
mkdir
(
os
.
path
.
join
(
sample_eggs
,
'index'
))
test
.
globs
[
'sample_eggs'
]
=
sample_eggs
...
...
src/zc/buildout/update.txt
View file @
6107db1f
...
...
@@ -81,14 +81,13 @@ new versions found in new releases:
Our buildout script has been updated to use the new eggs:
>>> cat(sample_buildout, 'bin', 'buildout')
#!/usr/local/bin/python2.4
-S
#!/usr/local/bin/python2.4
<BLANKLINE>
import sys
sys.path[0:0] = [
'/sample-buildout/eggs/zc.buildout-99.99-py2.4.egg',
'/sample-buildout/eggs/setuptools-99.99-py2.4.egg',
]
<BLANKLINE>
'/sample-buildout/eggs/zc.buildout-99.99-py2.4.egg',
'/sample-buildout/eggs/setuptools-99.99-py2.4.egg',
]
<BLANKLINE>
import zc.buildout.buildout
<BLANKLINE>
...
...
src/zc/buildout/upgrading_distribute.txt
0 → 100644
View file @
6107db1f
Installing setuptools/distribute
--------------------------------
Some initial test setup:
>>> import sys
>>> import zc.buildout
>>> dest = tmpdir('sample-install')
Setuptools (0.6something) is packaged as an ``.egg``. So when installing it,
the egg is downloaded and used. Distribute is packaged as a tarball, which
makes an easy_install call necessary. In older versions of buildout, the
``_call_easy_install()`` method would call ``_get_dist()`` to get hold of the
setuptools path for calling easy_install. When an updated "distribute" was
found, this would try an install again, leading to an infinite recursion.
The solution is to just use the setuptools location found at import time, like
happens with the buildout and setuptools location that is inserted in scripts'
paths.
We test this corner case by patching the ``_get_dist()`` call:
>>> def mock_get_dist(requirement, ws, always_unzip):
... raise RuntimeError("We should not get called")
When installing setuptools itself, we expect the "Getting dist" message not to
be printed. We call ``_call_easy_install()`` directly and get an error
because of a non-existing tarball, but that's the OK for this corner case
test: we only want to test that ``_get_dist()`` isn't getting called:
>>> class MockDist(object):
... def __str__(self):
... return 'nonexisting.tgz'
... @property
... def project_name(self):
... # Testing corner case: there *is* actually
... # a newer setuptools package on pypi than we
... # are running with, so it really installs it
... # and compares project_name. We're past the
... # point that we're testing, so we just raise
... # the normally expected error.
... raise zc.buildout.UserError(
... "Couldn't install: nonexisting.tgz")
>>> dist = MockDist()
>>> installer = zc.buildout.easy_install.Installer(
... dest=dest,
... links=[link_server],
... index=link_server+'index/',
... executable=sys.executable,
... always_unzip=True)
>>> installer._get_dist = mock_get_dist
>>> installer._call_easy_install('setuptools', None, dest, dist)
Traceback (most recent call last):
...
UserError: Couldn't install: nonexisting.tgz
z3c.recipe.scripts_/CHANGES.txt
deleted
100644 → 0
View file @
337b7586
Change History
**************
1.0.0
=====
Initial public version.
z3c.recipe.scripts_/README.txt
deleted
100644 → 0
View file @
337b7586
********************************
Buildout Script Recipe
********************************
.. contents::
The script recipe installs eggs into a buildout eggs directory, exactly
like zc.recipe.egg, and then generates scripts in a buildout bin
directory with egg paths baked into them.
z3c.recipe.scripts_/setup.py
deleted
100644 → 0
View file @
337b7586
##############################################################################
#
# Copyright (c) 2007 Zope Corporation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
##############################################################################
"""Setup for z3c.recipe.scripts package
$Id: setup.py 106736 2009-12-18 02:33:08Z gary $
"""
version
=
'1.0'
import
os
from
setuptools
import
setup
,
find_packages
def
read
(
*
rnames
):
return
open
(
os
.
path
.
join
(
os
.
path
.
dirname
(
__file__
),
*
rnames
)).
read
()
name
=
"z3c.recipe.scripts"
setup
(
name
=
name
,
version
=
version
,
author
=
"Gary Poster"
,
author_email
=
"gary.poster@canonical.com"
,
description
=
"Recipe for installing Python scripts"
,
long_description
=
(
read
(
'README.txt'
)
+
'
\
n
'
+
read
(
'CHANGES.txt'
)
+
'
\
n
'
+
'Detailed Documentation
\
n
'
'**********************
\
n
'
+
'
\
n
'
+
read
(
'src'
,
'z3c'
,
'recipe'
,
'scripts'
,
'README.txt'
)
+
'
\
n
'
+
'Download
\
n
'
'*********
\
n
'
),
keywords
=
"development build"
,
classifiers
=
[
'Development Status :: 5 - Production/Stable'
,
'Framework :: Buildout'
,
'Intended Audience :: Developers'
,
'License :: OSI Approved :: Zope Public License'
,
'Topic :: Software Development :: Build Tools'
,
'Topic :: Software Development :: Libraries :: Python Modules'
,
],
url
=
'http://cheeseshop.python.org/pypi/z3c.recipe.scripts'
,
license
=
"ZPL 2.1"
,
packages
=
find_packages
(
'src'
),
package_dir
=
{
''
:
'src'
},
namespace_packages
=
[
'z3c'
,
'z3c.recipe'
],
install_requires
=
[
'zc.buildout >=1.2.0'
,
'zc.recipe.egg'
,
'setuptools'
],
tests_require
=
[
'zope.testing'
],
test_suite
=
name
+
'.tests.test_suite'
,
entry_points
=
{
'zc.buildout'
:
[
'default = %s:Scripts'
%
name
,
'script = %s:Scripts'
%
name
,
'scripts = %s:Scripts'
%
name
,
'interpreter = %s:Interpreter'
%
name
,
]
},
include_package_data
=
True
,
zip_safe
=
False
,
)
z3c.recipe.scripts_/src/z3c/__init__.py
deleted
100644 → 0
View file @
337b7586
__import__
(
'pkg_resources'
).
declare_namespace
(
__name__
)
z3c.recipe.scripts_/src/z3c/recipe/__init__.py
deleted
100644 → 0
View file @
337b7586
__import__
(
'pkg_resources'
).
declare_namespace
(
__name__
)
z3c.recipe.scripts_/src/z3c/recipe/scripts/README.txt
deleted
100644 → 0
View file @
337b7586
Script and interpreter generation
=================================
This recipe is very similar to zc.recipe.egg, and if you are familiar with its
options, you will be able to use this one easily.
The script and interpreter generation in this recipe are improved from
those provided by zc.recipe.egg in two basic ways.
- The interpreter generated by the script supports all interpreter
options, as opposed to the subset provided by zc.recipe.egg.
- Both scripts and interpreters from this recipe can optionally choose
to include site-packages, and even sitecustomize.
The recipe takes several options. First, here's the list of the options
that overlap from the standard zc.recipe.eggs scripts recipe. After
this, we'll list the new options and describe them.
* eggs
* find-links
* index
* python
* extra-paths
* entry-points
* scripts
* dependent-scripts
* interpreter
* arguments
* initialization
* relative-paths
In addition to these, the recipe offers these new options. They are
introduced here, and described more in depth below.
add-site-packages
You can choose to have the site-packages of the underlying Python
available to your script or interpreter, in addition to the packages
from your eggs. See the section on this option for motivations and
warnings.
extends
You can extend another section using this value. It is intended to be
used by extending a section that uses this package's scripts recipe.
In this manner, you can avoid repeating yourself.
exec-sitecustomize
Normally the Python's real sitecustomize module is not processed.
If you want it to be processed, set this value to 'true'. This will
be honored irrespective of the setting for include-site-paths.
script-initialization
The standard initialization code affects both an interpreter and scripts.
The code in script-initialization is used only for the generated scripts.
Finally, the "interpreter" entry point ignores ``script-initialization``,
``scripts``, and ``arguments``, and provides yet another additional option.
name
While, by default, the interpreter recipe takes the name of the
section to be the desired interpreter name, you can specify the
interpreter name here instead.
Script generation
-----------------
Generating a basic script looks virtually identical to using zc.recipe.egg.
(Note that the find-links and index values are typically not needed; they
are included to help make this document run as a test successfully.)
>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... parts = demo
...
... [demo]
... recipe = z3c.recipe.scripts
... eggs = demo<0.3
... find-links = %(server)s
... index = %(server)s/index
... """ % dict(server=link_server))
>>> print system(buildout),
Installing demo.
Getting distribution for 'demo<0.3'.
Got demo 0.2.
Getting distribution for 'demoneeded'.
Got demoneeded 1.2c1.
Generated script '/sample-buildout/bin/demo'.
>>> print system(join(sample_buildout, 'bin', 'demo')),
2 2
Interpreter generation
----------------------
As with zc.recipe.egg, you can generate an interpreter with the default
script recipe shown above by supplying the "interpreter" option.
This example will create both an entry point script and an interpreter.
>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... parts = demo
...
... [demo]
... recipe = z3c.recipe.scripts
... eggs = demo<0.3
... find-links = %(server)s
... index = %(server)s/index
... interpreter = py
... """ % dict(server=link_server))
>>> print system(buildout),
Uninstalling demo.
Installing demo.
Generated script '/sample-buildout/bin/demo'.
Generated interpreter '/sample-buildout/bin/py'.
You can also generate an interpreter alone with the ``interpreter`` recipe.
>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... parts = py
...
... [py]
... recipe = z3c.recipe.scripts:interpreter
... eggs = demo<0.3
... find-links = %(server)s
... index = %(server)s/index
... """ % dict(server=link_server))
>>> print system(buildout),
Uninstalling demo.
Installing py.
Generated interpreter '/sample-buildout/bin/py'.
In both cases, the bin/py script works by restarting Python after
specifying a special path in PYTHONPATH. This example shows the UNIX version;
the Windows version actually uses subprocess instead.
>>> cat(sample_buildout, 'bin', 'py') # doctest: +NORMALIZE_WHITESPACE
#!/usr/bin/python2.4 -S
<BLANKLINE>
import os
import sys
<BLANKLINE>
argv = [sys.executable] + sys.argv[1:]
environ = os.environ.copy()
path = '/sample-buildout/parts/py'
if environ.get('PYTHONPATH'):
path = os.pathsep.join([path, environ['PYTHONPATH']])
environ['PYTHONPATH'] = path
os.execve(sys.executable, argv, environ)
The path is a directory that contains two files: our own site.py and
sitecustomize.py. The site.py is modified from the underlying Python's
site.py, and is responsible for setting up our paths. The
sitecustomize.py is responsible for running the initialization code
provided.
>>> ls(sample_buildout, 'parts', 'py')
- site.py
- sitecustomize.py
Here's an example of using the generated interpreter.
>>> print system(join(sample_buildout, 'bin', 'py') +
... ' -c "import sys, pprint; pprint.pprint(sys.path[:3])"')
['',
'/sample-buildout/eggs/demo-0.2-pyN.N.egg',
'/sample-buildout/eggs/demoneeded-1.2c1-pyN.N.egg']
<BLANKLINE>
Including site-packages and sitecustomize
-----------------------------------------
As introduced above, this recipe supports including site packages. This has
some advantages and some serious dangers.
A typical reason to include site-packages is that it is easier to
install one or more dependencies in your Python than it is with
buildbot. Some packages, such as lxml or Python PostgreSQL integration,
have dependencies that can be much easier to build and/or install using
other mechanisms, such as your operating system's package manager. By
installing some core packages into your Python's site-packages, this can
significantly simplify some application installations.
However, doing this has a significant danger. One of the primary goals
of buildout is to provide repeatability. Some packages (one of the
better known Python openid packages, for instance) change their behavior
depending on what packages are available. If Python curl bindings are
available, these may be preferred by the library. If a certain XML
package is installed, it may be preferred by the library. These hidden
choices may cause small or large behavior differences. The fact that
they can be rarely encountered can actually make it worse: you forget
that this might be a problem, and debugging the differences can be
difficult. If you allow site-packages to be included in your buildout,
and the Python you use is not managed precisely by your application (for
instance, it is a system Python), you open yourself up to these
possibilities. Don't be unaware of the dangers.
To show off these features, we need to use buildout with a Python
executable with some extra paths to show ``add-site-packages``; and one
guaranteed to have a sitecustomize module to show
``exec-sitecustomize``. We'll make one using a test fixture called
``make_py``. The os.environ change below will go into the sitecustomize,
and the site_packages_path will be in the Python's path.
>>> py_path, site_packages_path = make_py(initialization='''\
... import os
... os.environ['zc.buildout'] = 'foo bar baz shazam'
... ''')
>>> print site_packages_path
/executable_buildout/site-packages
Now let's take a look at add-site-packages.
>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... parts = py
... executable = %(py_path)s
...
... [py]
... recipe = z3c.recipe.scripts:interpreter
... add-site-packages = true
... eggs = demo<0.3
... find-links = %(server)s
... index = %(server)s/index
... """ % dict(server=link_server, py_path=py_path))
>>> print system(buildout),
Uninstalling py.
Installing py.
Generated interpreter '/sample-buildout/bin/py'.
>>> print system(join(sample_buildout, 'bin', 'py') +
... ''' -c "import sys, pprint; pprint.pprint(sys.path)"''')
... # doctest: +ELLIPSIS
['',
'/sample-buildout/eggs/demo-0.2-pyN.N.egg',
'/sample-buildout/eggs/demoneeded-1.2c1-pyN.N.egg',
'/executable_buildout/eggs/setuptools-0.6c11-pyN.N.egg',
'/executable_buildout/site-packages',
'/sample-buildout/parts/py',
'/executable_buildout/parts/py',
...]
<BLANKLINE>
Next we will use the exec-sitecustomize option. It simply copies
Python's underlying sitecustomize module, if it exists, to the local
version. The os.environ change shown above in the make_py call will go
into the sitecustomize.
>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... parts = py
... executable = %(py_path)s
...
... [py]
... recipe = z3c.recipe.scripts:interpreter
... exec-sitecustomize = true
... eggs = demo<0.3
... find-links = %(server)s
... index = %(server)s/index
... """ % dict(server=link_server, py_path=py_path))
>>> print system(buildout),
Uninstalling py.
Installing py.
Generated interpreter '/sample-buildout/bin/py'.
>>> cat(sample_buildout, 'parts', 'py', 'sitecustomize.py')
... # doctest: +NORMALIZE_WHITESPACE
<BLANKLINE>
# The following is from
# /executable_buildout/parts/py/sitecustomize.py
<BLANKLINE>
import os
os.environ['zc.buildout'] = 'foo bar baz shazam'
>>> print system(join(sample_buildout, 'bin', 'py') +
... ''' -c "import os; print os.environ['zc.buildout']"''')
foo bar baz shazam
<BLANKLINE>
Options
-------
We'll focus now on the options that are different than zc.recipe.egg.
Let's look at the ``extends`` option first.
>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... parts = demo python
...
... [demo]
... recipe = z3c.recipe.scripts
... eggs = demo<0.3
... find-links = %(server)s
... index = %(server)s/index
...
... [python]
... recipe = z3c.recipe.scripts:interpreter
... extends = demo
... initialization =
... import os
... os.environ['zc.buildout'] = 'foo bar baz shazam'
... """ % dict(server=link_server))
That makes it easier to specify some initialization for the interpreter
that is different than a script, while duplicating other configuration.
Now let's put it in action.
>>> print system(buildout),
Uninstalling py.
Installing demo.
Generated script '/sample-buildout/bin/demo'.
Installing python.
Generated interpreter '/sample-buildout/bin/python'.
>>> print system(join(sample_buildout, 'bin', 'python') +
... ' -c "import sys, pprint; pprint.pprint(sys.path[:3])"')
['',
'/sample-buildout/eggs/demo-0.2-pyN.N.egg',
'/sample-buildout/eggs/demoneeded-1.2c1-pyN.N.egg']
<BLANKLINE>
>>> print system(join(sample_buildout, 'bin', 'python') +
... ''' -c "import os; print os.environ['zc.buildout']"'''),
foo bar baz shazam
Note that the parts/py directory has been cleaned up, and parts/python has
been created.
>>> ls(sample_buildout, 'parts')
d demo
d python
If you want to have initialization that only affects scripts, not the
interpreter, you can use script-initialization. Here's a demonstration.
>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... parts = demo
...
... [demo]
... recipe = z3c.recipe.scripts
... eggs = demo<0.3
... find-links = %(server)s
... index = %(server)s/index
... interpreter = py
... script-initialization =
... print "Hi from the script"
... """ % dict(server=link_server))
>>> print system(buildout),
Uninstalling python.
Uninstalling demo.
Installing demo.
Generated script '/sample-buildout/bin/demo'.
Generated interpreter '/sample-buildout/bin/py'.
>>> print system(join(sample_buildout, 'bin', 'py') +
... ''' -c "print 'Hi from the interpreter'"'''),
Hi from the interpreter
>>> print system(join(sample_buildout, 'bin', 'demo')),
Hi from the script
2 2
The last new option is ``name``. This simply changes the name of the
interpreter, so that you are not forced to use the name of the section.
>>> write(sample_buildout, 'buildout.cfg',
... """
... [buildout]
... parts = interpreter
...
... [interpreter]
... name = python2
... recipe = z3c.recipe.scripts:interpreter
... eggs = demo<0.3
... find-links = %(server)s
... index = %(server)s/index
... """ % dict(server=link_server))
>>> print system(buildout),
Uninstalling demo.
Installing interpreter.
Generated interpreter '/sample-buildout/bin/python2'.
>>> print system(join(sample_buildout, 'bin', 'python2') +
... ' -c "print 42"')
42
<BLANKLINE>
The other options all identical to the zc.recipe.egg script. Here are some
quick demos and discussions.
z3c.recipe.scripts_/src/z3c/recipe/scripts/__init__.py
deleted
100644 → 0
View file @
337b7586
from
z3c.recipe.scripts.scripts
import
Scripts
,
Interpreter
z3c.recipe.scripts_/src/z3c/recipe/scripts/scripts.py
deleted
100644 → 0
View file @
337b7586
##############################################################################
#
# Copyright (c) 2009-2010 Zope Corporation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
##############################################################################
"""Install scripts from eggs.
"""
import
os
import
zc.buildout
import
zc.buildout.easy_install
from
zc.recipe.egg.egg
import
ScriptBase
class
Base
(
ScriptBase
):
def
__init__
(
self
,
buildout
,
name
,
options
):
if
'extends'
in
options
:
options
.
update
(
buildout
[
options
[
'extends'
]])
super
(
Base
,
self
).
__init__
(
buildout
,
name
,
options
)
self
.
default_eggs
=
''
# Disables feature from zc.recipe.egg.
b_options
=
buildout
[
'buildout'
]
options
[
'parts-directory'
]
=
os
.
path
.
join
(
b_options
[
'parts-directory'
],
self
.
name
)
value
=
options
.
setdefault
(
'add-site-packages'
,
b_options
.
get
(
'add-site-packages'
,
'false'
))
if
value
not
in
(
'true'
,
'false'
):
raise
zc
.
buildout
.
UserError
(
"Invalid value for add-site-packages option: %s"
%
(
value
,))
self
.
add_site_packages
=
(
value
==
'true'
)
value
=
options
.
setdefault
(
'exec-sitecustomize'
,
b_options
.
get
(
'exec-sitecustomize'
,
'false'
))
if
value
not
in
(
'true'
,
'false'
):
raise
zc
.
buildout
.
UserError
(
"Invalid value for exec-sitecustomize option: %s"
%
(
value
,))
self
.
exec_sitecustomize
=
(
value
==
'true'
)
class
Interpreter
(
Base
):
def
__init__
(
self
,
buildout
,
name
,
options
):
super
(
Interpreter
,
self
).
__init__
(
buildout
,
name
,
options
)
options
.
setdefault
(
'name'
,
name
)
def
install
(
self
):
reqs
,
ws
=
self
.
working_set
()
options
=
self
.
options
generated
=
[]
if
not
os
.
path
.
exists
(
options
[
'parts-directory'
]):
os
.
mkdir
(
options
[
'parts-directory'
])
generated
.
append
(
options
[
'parts-directory'
])
generated
.
extend
(
zc
.
buildout
.
easy_install
.
generate_scripts
(
options
[
'bin-directory'
],
ws
,
options
[
'executable'
],
options
[
'parts-directory'
],
interpreter
=
options
[
'name'
],
extra_paths
=
self
.
extra_paths
,
initialization
=
options
.
get
(
'initialization'
,
''
),
add_site_packages
=
self
.
add_site_packages
,
exec_sitecustomize
=
self
.
exec_sitecustomize
,
relative_paths
=
self
.
_relative_paths
,
))
return
generated
update
=
install
class
Scripts
(
Base
):
def
_install
(
self
,
reqs
,
ws
,
scripts
):
options
=
self
.
options
generated
=
[]
if
not
os
.
path
.
exists
(
options
[
'parts-directory'
]):
os
.
mkdir
(
options
[
'parts-directory'
])
generated
.
append
(
options
[
'parts-directory'
])
generated
.
extend
(
zc
.
buildout
.
easy_install
.
generate_scripts
(
options
[
'bin-directory'
],
ws
,
options
[
'executable'
],
options
[
'parts-directory'
],
reqs
=
reqs
,
scripts
=
scripts
,
interpreter
=
options
.
get
(
'interpreter'
),
extra_paths
=
self
.
extra_paths
,
initialization
=
options
.
get
(
'initialization'
,
''
),
add_site_packages
=
self
.
add_site_packages
,
exec_sitecustomize
=
self
.
exec_sitecustomize
,
relative_paths
=
self
.
_relative_paths
,
script_arguments
=
options
.
get
(
'arguments'
,
''
),
script_initialization
=
options
.
get
(
'script-initialization'
,
''
)
))
return
generated
z3c.recipe.scripts_/src/z3c/recipe/scripts/tests.py
deleted
100644 → 0
View file @
337b7586
##############################################################################
#
# Copyright (c) 2006 Zope Corporation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
##############################################################################
import
os
,
re
,
shutil
,
sys
import
zc.buildout.tests
import
zc.buildout.testselectingpython
import
zc.buildout.testing
import
unittest
from
zope.testing
import
doctest
,
renormalizing
# We do not explicitly test the recipe support for the ``eggs``,
# ``find-links``, and ``index`` options because they are used for most or
# all of the examples. The README tests ``extends``,
# ``include-site-customization`` and ``name``. That leaves ``python``,
# ``extra-paths``, ``initialization``, ``relative-paths``, and
# ``include-site-packages``.
def
supports_python_option
():
"""
This simply shows that the ``python`` option can specify another section to
find the ``executable``. (The ``python`` option defaults to looking in the
``buildout`` section.) We do this by creating a custom Python that will have
some initialization that we can look for.
>>> py_path, site_packages_path = make_py(initialization='''
... import os
... os.environ['zc.buildout'] = 'foo bar baz shazam'
... ''')
>>> write(sample_buildout, 'buildout.cfg',
... '''
... [buildout]
... parts = py
...
... [custom_python]
... executable = %(py_path)s
...
... [py]
... recipe = z3c.recipe.scripts:interpreter
... exec-sitecustomize = true
... eggs = demo<0.3
... find-links = %(server)s
... index = %(server)s/index
... python = custom_python
... ''' % dict(server=link_server, py_path=py_path))
>>> print system(buildout),
Installing py.
Getting distribution for 'demo<0.3'.
Got demo 0.2.
Getting distribution for 'demoneeded'.
Got demoneeded 1.2c1.
Generated interpreter '/sample-buildout/bin/py'.
>>> print system(join(sample_buildout, 'bin', 'py') +
... ''' -c "import os; print os.environ['zc.buildout']"'''),
foo bar baz shazam
"""
def
interpreter_recipe_supports_extra_paths_option
():
"""
This shows that specifying extra-paths will affect sys.path.
This recipe will not add paths that do not exist, so we create them.
>>> mkdir(sample_buildout, 'foo')
>>> mkdir(sample_buildout, 'foo', 'bar')
>>> mkdir(sample_buildout, 'spam')
>>> write(sample_buildout, 'buildout.cfg',
... '''
... [buildout]
... parts = py
...
... [py]
... recipe = z3c.recipe.scripts:interpreter
... find-links = %(server)s
... index = %(server)s/index
... extra-paths =
... ${buildout:directory}/foo/bar
... ${buildout:directory}/spam
... ''' % dict(server=link_server))
>>> print system(buildout),
Installing py.
Generated interpreter '/sample-buildout/bin/py'.
>>> print system(join(sample_buildout, 'bin', 'py') +
... ''' -c "import sys;print 'path' + ' '.join(sys.path)"''')
... # doctest:+ELLIPSIS
path.../foo/bar /sample-buildout/spam...
"""
def
interpreter_recipe_supports_initialization_option
():
"""
This simply shows that the ``initialization`` option can specify code to
run on initialization.
>>> write(sample_buildout, 'buildout.cfg',
... '''
... [buildout]
... parts = py
...
... [py]
... recipe = z3c.recipe.scripts:interpreter
... initialization =
... import os
... os.environ['zc.buildout'] = 'foo bar baz shazam'
... eggs = demo<0.3
... find-links = %(server)s
... index = %(server)s/index
... ''' % dict(server=link_server))
>>> print system(buildout),
Installing py.
Getting distribution for 'demo<0.3'.
Got demo 0.2.
Getting distribution for 'demoneeded'.
Got demoneeded 1.2c1.
Generated interpreter '/sample-buildout/bin/py'.
>>> cat(sample_buildout, 'parts', 'py', 'sitecustomize.py')
... # doctest: +NORMALIZE_WHITESPACE
<BLANKLINE>
import os
os.environ['zc.buildout'] = 'foo bar baz shazam'
>>> print system(join(sample_buildout, 'bin', 'py') +
... ''' -c "import os; print os.environ['zc.buildout']"'''),
foo bar baz shazam
This also works with the exec-sitecustomize option, processing local
initialization, and then the Python's initialization. We show this with a
custom Python.
>>> py_path, site_packages_path = make_py(initialization='''
... import os
... os.environ['zc.buildout'] = 'foo bar baz shazam'
... ''')
>>> write(sample_buildout, 'buildout.cfg',
... '''
... [buildout]
... parts = py
...
... [custom_python]
... executable = %(py_path)s
...
... [py]
... recipe = z3c.recipe.scripts:interpreter
... initialization =
... import os
... os.environ['zc.recipe.egg'] = 'baLOOba'
... exec-sitecustomize = true
... eggs = demo<0.3
... find-links = %(server)s
... index = %(server)s/index
... python = custom_python
... ''' % dict(server=link_server, py_path=py_path))
>>> print system(buildout),
Uninstalling py.
Installing py.
Generated interpreter '/sample-buildout/bin/py'.
>>> cat(sample_buildout, 'parts', 'py', 'sitecustomize.py')
... # doctest: +NORMALIZE_WHITESPACE
<BLANKLINE>
import os
os.environ['zc.recipe.egg'] = 'baLOOba'
<BLANKLINE>
# The following is from
# /executable_buildout/parts/py/sitecustomize.py
<BLANKLINE>
import os
os.environ['zc.buildout'] = 'foo bar baz shazam'
>>> print system(join(sample_buildout, 'bin', 'py') + ' -c ' +
... '''"import os; print os.environ['zc.recipe.egg']"'''),
baLOOba
>>> print system(join(sample_buildout, 'bin', 'py') +
... ''' -c "import os; print os.environ['zc.buildout']"'''),
foo bar baz shazam
"""
def
interpreter_recipe_supports_relative_paths_option
():
"""
This shows that the relative-paths option affects the code for inserting
paths into sys.path.
>>> write(sample_buildout, 'buildout.cfg',
... '''
... [buildout]
... parts = py
...
... [py]
... recipe = z3c.recipe.scripts:interpreter
... find-links = %(server)s
... index = %(server)s/index
... relative-paths = true
... extra-paths =
... /foo/bar
... ${buildout:directory}/spam
... ''' % dict(server=link_server))
>>> print system(buildout),
Installing py.
Generated interpreter '/sample-buildout/bin/py'.
Let's look at the site.py that was generated:
>>> import sys
>>> sys.stdout.write('#'); cat(sample_buildout, 'parts', 'py', 'site.py')
... # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS
#...
def addsitepackages(known_paths):
paths = []
<BLANKLINE>
import os
<BLANKLINE>
join = os.path.join
base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
base = os.path.dirname(base)
base = os.path.dirname(base)
paths[0:0] = [ # eggs
'/foo/bar',
join(base, 'spam')
]...
"""
def
setUp
(
test
):
zc
.
buildout
.
tests
.
easy_install_SetUp
(
test
)
zc
.
buildout
.
testing
.
install_develop
(
'zc.recipe.egg'
,
test
)
zc
.
buildout
.
testing
.
install_develop
(
'z3c.recipe.scripts'
,
test
)
def
setUpSelecting
(
test
):
zc
.
buildout
.
testselectingpython
.
setup
(
test
)
zc
.
buildout
.
testing
.
install_develop
(
'zc.recipe.egg'
,
test
)
zc
.
buildout
.
testing
.
install_develop
(
'z3c.recipe.scripts'
,
test
)
def
test_suite
():
suite
=
unittest
.
TestSuite
((
doctest
.
DocFileSuite
(
'README.txt'
,
setUp
=
setUp
,
tearDown
=
zc
.
buildout
.
testing
.
buildoutTearDown
,
checker
=
renormalizing
.
RENormalizing
([
zc
.
buildout
.
testing
.
normalize_path
,
zc
.
buildout
.
testing
.
normalize_endings
,
zc
.
buildout
.
testing
.
normalize_script
,
zc
.
buildout
.
testing
.
normalize_egg_py
,
zc
.
buildout
.
tests
.
normalize_bang
,
(
re
.
compile
(
r'zc.buildout(-\
S+)?[.]egg(-li
nk)?'
),
'zc.buildout.egg'
),
(
re
.
compile
(
'[-d] setuptools-[^-]+-'
),
'setuptools-X-'
),
(
re
.
compile
(
r'setuptools-[\
w.]+-py
'), '
setuptools
-
X
-
py
'),
(re.compile(r'
eggs
\\\\
demo
'), '
eggs
/
demo
'),
(re.compile(r'
[
a
-
zA
-
Z
]:
\\\\
foo
\\\\
bar
'), '
/
foo
/
bar
'),
(re.compile(r'
\
#!\S+\bpython\S*'), '#!/usr/bin/python'),
# Normalize generate_script's Windows interpreter to UNIX:
(
re
.
compile
(
r'\nimport subprocess\n'
),
'
\
n
'
),
(
re
.
compile
(
'subprocess
\
\
.call
\
\
(argv, env=environ
\
\
)'
),
'os.execve(sys.executable, argv, environ)'
),
])
),
doctest
.
DocTestSuite
(
setUp
=
setUp
,
tearDown
=
zc
.
buildout
.
testing
.
buildoutTearDown
,
checker
=
renormalizing
.
RENormalizing
([
zc
.
buildout
.
testing
.
normalize_path
,
zc
.
buildout
.
testing
.
normalize_endings
,
zc
.
buildout
.
testing
.
normalize_egg_py
,
(
re
.
compile
(
r'[a-zA-Z]:\\\\foo\\\\bar'
),
'/foo/bar'
),
]),
),
))
return
suite
if
__name__
==
'__main__'
:
unittest
.
main
(
defaultTest
=
'test_suite'
)
zc.recipe.egg_/src/zc/recipe/egg/README.txt
View file @
6107db1f
...
...
@@ -46,7 +46,7 @@ We have a link server that has a number of distributions:
<a href="other-1.0-py2.3.egg">other-1.0-py2.3.egg</a><br>
</body></html>
We have a sample buildout. Let's update its configuration file to
We have a sample buildout. Let's update it
'
s configuration file to
install the demo package.
>>> write(sample_buildout, 'buildout.cfg',
...
...
@@ -154,8 +154,6 @@ dependent-scripts
interpreter
The name of a script to generate that allows access to a Python
interpreter that has the path set based on the eggs installed.
(See the ``z3c.recipe.scripts`` recipe for a more full-featured
interpreter.)
extra-paths
Extra paths to include in a generated script.
...
...
@@ -189,7 +187,7 @@ Let's add an interpreter option:
... interpreter = py-demo
... """ % dict(server=link_server))
Note that we omitted the entry point name from the recipe
Note that we om
m
itted the entry point name from the recipe
specification. We were able to do this because the scripts recipe is
the default entry point for the zc.recipe.egg egg.
...
...
@@ -375,7 +373,7 @@ extra-paths option:
Let's look at the script that was generated:
>>> cat(sample_buildout, 'bin', 'foo') # doctest: +NORMALIZE_WHITESPACE
#!/usr/local/bin/python2.4
-S
#!/usr/local/bin/python2.4
<BLANKLINE>
import sys
sys.path[0:0] = [
...
...
@@ -385,7 +383,6 @@ Let's look at the script that was generated:
'/sample-buildout/spam',
]
<BLANKLINE>
<BLANKLINE>
import eggrecipedemo
<BLANKLINE>
if __name__ == '__main__':
...
...
@@ -423,7 +420,7 @@ breaking scripts.
Let's look at the script that was generated:
>>> cat(sample_buildout, 'bin', 'foo') # doctest: +NORMALIZE_WHITESPACE
#!/usr/local/bin/python2.4
-S
#!/usr/local/bin/python2.4
<BLANKLINE>
import os
<BLANKLINE>
...
...
@@ -470,7 +467,7 @@ each individual script section:
Generated script '/sample-buildout/bin/foo'.
>>> cat(sample_buildout, 'bin', 'foo') # doctest: +NORMALIZE_WHITESPACE
#!/usr/local/bin/python2.4
-S
#!/usr/local/bin/python2.4
<BLANKLINE>
import os
<BLANKLINE>
...
...
@@ -523,7 +520,7 @@ to be included in generated scripts:
Generated script '/sample-buildout/bin/foo'.
>>> cat(sample_buildout, 'bin', 'foo') # doctest: +NORMALIZE_WHITESPACE
#!/usr/local/bin/python2.4
-S
#!/usr/local/bin/python2.4
<BLANKLINE>
import sys
sys.path[0:0] = [
...
...
@@ -581,16 +578,15 @@ declare entry points using the entry-points option:
- other
>>> cat(sample_buildout, 'bin', 'other')
#!/usr/local/bin/python2.4
-S
#!/usr/local/bin/python2.4
<BLANKLINE>
import sys
sys.path[0:0] = [
'/sample-buildout/eggs/demo-0.4c1-py2.4.egg',
'/sample-buildout/eggs/demoneeded-1.2c1-py2.4.egg',
'/foo/bar',
'/sample-buildout/spam',
]
<BLANKLINE>
'/sample-buildout/eggs/demo-0.4c1-py2.4.egg',
'/sample-buildout/eggs/demoneeded-1.2c1-py2.4.egg',
'/foo/bar',
'/sample-buildout/spam',
]
<BLANKLINE>
import foo.bar
<BLANKLINE>
...
...
@@ -644,4 +640,3 @@ be made to contact an index server:
Uninstalling bigdemo.
Installing demo.
Generated script '/sample-buildout/bin/foo'.
zc.recipe.egg_/src/zc/recipe/egg/api.txt
View file @
6107db1f
...
...
@@ -15,7 +15,7 @@ To illustrate, we create a sample recipe that is a very thin layer
around the egg recipe:
>>> mkdir(sample_buildout, 'sample')
>>> write(sample_buildout, 'sample', 'sample.py',
>>> write(sample_buildout, 'sample', 'sample.py',
... """
... import logging, os
... import zc.recipe.egg
...
...
@@ -53,7 +53,7 @@ of extra requirements to be included in the working set.
>>> write(sample_buildout, 'sample', 'setup.py',
... """
... from setuptools import setup
...
...
... setup(
... name = "sample",
... entry_points = {'zc.buildout': ['default = sample:Sample']},
...
...
@@ -95,13 +95,12 @@ We can see that the options were augmented with additional data
computed by the egg recipe by looking at .installed.cfg:
>>> cat(sample_buildout, '.installed.cfg')
... # doctest: +NORMALIZE_WHITESPACE
[buildout]
installed_develop_eggs = /sample-buildout/develop-eggs/sample.egg-link
parts = sample-part
<BLANKLINE>
[sample-part]
__buildout_installed__ =
__buildout_installed__ =
__buildout_signature__ = sample-6aWMvV2EJ9Ijq+bR8ugArQ==
zc.recipe.egg-cAsnudgkduAa/Fd+WJIM6Q==
setuptools-0.6-py2.4.egg
...
...
@@ -117,7 +116,6 @@ computed by the egg recipe by looking at .installed.cfg:
extras = other
find-links = http://localhost:27071/
index = http://localhost:27071/index
python = buildout
recipe = sample
If we use the extra-paths option:
...
...
zc.recipe.egg_/src/zc/recipe/egg/custom.txt
View file @
6107db1f
...
...
@@ -50,7 +50,7 @@ compiler
swig
The path to the swig executable
swig-cpp
swig-cpp
Make SWIG create C++ files (default is C)
swig-opts
...
...
@@ -73,14 +73,14 @@ index
alternate index with this option. If you use the links option and
if the links point to the needed distributions, then the index can
be anything and will be largely ignored. In the examples, here,
we'll just point to an empty directory on our link server. This
we'll just point to an empty directory on our link server. This
will make our examples run a little bit faster.
python
The name of a section to get the Python executable from.
If not specified, then the buildout python option is used. The
Python executable is found in the executable option of the named
section.
section.
environment
The name of a section with additional environment variables. The
...
...
@@ -150,7 +150,6 @@ eggs directory can be shared across multiple buildouts.
>>> ls(sample_buildout, 'develop-eggs')
d extdemo-1.4-py2.4-unix-i686.egg
- z3c.recipe.scripts.egg-link
- zc.recipe.egg.egg-link
Note that no scripts or dependencies are installed. To install
...
...
@@ -189,7 +188,7 @@ Let's define a script that uses out ext demo:
...
... [demo]
... recipe = zc.recipe.egg
... eggs = demo
... eggs = demo
... extdemo
... entry-points = demo=demo:main
... """ % dict(server=link_server))
...
...
@@ -232,7 +231,6 @@ We won't get an update.
>>> ls(sample_buildout, 'develop-eggs')
- demo.egg-link
d extdemo-1.4-py2.4-unix-i686.egg
- z3c.recipe.scripts.egg-link
- zc.recipe.egg.egg-link
But if we run the buildout in the default on-line and newest modes, we
...
...
@@ -250,7 +248,6 @@ version is imported:
- demo.egg-link
d extdemo-1.4-py2.4-linux-i686.egg
d extdemo-1.5-py2.4-linux-i686.egg
- z3c.recipe.scripts.egg-link
- zc.recipe.egg.egg-link
Controlling the version used
...
...
@@ -273,7 +270,7 @@ We can specify a specific version using the egg option:
...
... [demo]
... recipe = zc.recipe.egg
... eggs = demo
... eggs = demo
... extdemo ==1.4
... entry-points = demo=demo:main
... """ % dict(server=link_server))
...
...
@@ -290,7 +287,6 @@ We can specify a specific version using the egg option:
>>> ls(sample_buildout, 'develop-eggs')
- demo.egg-link
d extdemo-1.4-py2.4-linux-i686.egg
- z3c.recipe.scripts.egg-link
- zc.recipe.egg.egg-link
...
...
@@ -444,7 +440,7 @@ Create a clean buildout.cfg w/o the checkenv recipe, and delete the recipe:
Uninstalling extdemo.
Installing extdemo.
zip_safe flag not set; analyzing archive contents...
>>> rmdir(sample_buildout, 'recipes')
...
...
@@ -500,7 +496,7 @@ compiler
swig
The path to the swig executable
swig-cpp
swig-cpp
Make SWIG create C++ files (default is C)
swig-opts
...
...
@@ -510,7 +506,7 @@ python
The name of a section to get the Python executable from.
If not specified, then the buildout python option is used. The
Python executable is found in the executable option of the named
section.
section.
To illustrate this, we'll use a directory containing the extdemo
example from the earlier section:
...
...
@@ -536,7 +532,7 @@ example from the earlier section:
...
... [demo]
... recipe = zc.recipe.egg
... eggs = demo
... eggs = demo
... extdemo
... entry-points = demo=demo:main
... """ % dict(extdemo=extdemo))
...
...
@@ -557,7 +553,6 @@ Our develop-eggs now includes an egg link for extdemo:
>>> ls('develop-eggs')
- demo.egg-link
- extdemo.egg-link
- z3c.recipe.scripts.egg-link
- zc.recipe.egg.egg-link
and the extdemo now has a built extension:
...
...
zc.recipe.egg_/src/zc/recipe/egg/egg.py
View file @
6107db1f
...
...
@@ -19,12 +19,11 @@ $Id$
import
logging
,
os
,
re
,
zipfile
import
zc.buildout.easy_install
class
Eggs
(
object
):
def
__init__
(
self
,
buildout
,
name
,
options
):
self
.
buildout
=
buildout
self
.
name
=
self
.
default_eggs
=
name
self
.
name
=
name
self
.
options
=
options
b_options
=
buildout
[
'buildout'
]
links
=
options
.
get
(
'find-links'
,
b_options
[
'find-links'
])
...
...
@@ -53,7 +52,7 @@ class Eggs(object):
# verify that this is None, 'true' or 'false'
get_bool
(
options
,
'unzip'
)
python
=
options
.
setdefaul
t
(
'python'
,
b_options
[
'python'
])
python
=
options
.
ge
t
(
'python'
,
b_options
[
'python'
])
options
[
'executable'
]
=
buildout
[
python
][
'executable'
]
def
working_set
(
self
,
extra
=
()):
...
...
@@ -66,16 +65,15 @@ class Eggs(object):
distributions
=
[
r
.
strip
()
for
r
in
options
.
get
(
'eggs'
,
self
.
default_eggs
).
split
(
'
\
n
'
)
for
r
in
options
.
get
(
'eggs'
,
self
.
name
).
split
(
'
\
n
'
)
if
r
.
strip
()]
orig_distributions
=
distributions
[:]
distributions
.
extend
(
extra
)
if
b_options
.
get
(
'offline'
)
==
'true'
:
if
self
.
buildout
[
'buildout'
]
.
get
(
'offline'
)
==
'true'
:
ws
=
zc
.
buildout
.
easy_install
.
working_set
(
distributions
,
options
[
'executable'
],
[
options
[
'develop-eggs-directory'
],
options
[
'eggs-directory'
]],
[
options
[
'develop-eggs-directory'
],
options
[
'eggs-directory'
]]
)
else
:
kw
=
{}
...
...
@@ -87,7 +85,7 @@ class Eggs(object):
index
=
self
.
index
,
executable
=
options
[
'executable'
],
path
=
[
options
[
'develop-eggs-directory'
]],
newest
=
b_options
.
get
(
'newest'
)
==
'true'
,
newest
=
self
.
buildout
[
'buildout'
]
.
get
(
'newest'
)
==
'true'
,
allow_hosts
=
self
.
allow_hosts
,
**
kw
)
...
...
@@ -99,19 +97,16 @@ class Eggs(object):
update
=
install
class
ScriptBase
(
Eggs
):
class
Scripts
(
Eggs
):
def
__init__
(
self
,
buildout
,
name
,
options
):
super
(
ScriptBase
,
self
).
__init__
(
buildout
,
name
,
options
)
b_options
=
buildout
[
'buildout'
]
super
(
Scripts
,
self
).
__init__
(
buildout
,
name
,
options
)
options
[
'bin-directory'
]
=
b
_options
[
'bin-directory'
]
options
[
'bin-directory'
]
=
b
uildout
[
'buildout'
]
[
'bin-directory'
]
options
[
'_b'
]
=
options
[
'bin-directory'
]
# backward compat.
self
.
extra_paths
=
[
os
.
path
.
join
(
b
_options
[
'directory'
],
p
.
strip
())
os
.
path
.
join
(
b
uildout
[
'buildout'
]
[
'directory'
],
p
.
strip
())
for
p
in
options
.
get
(
'extra-paths'
,
''
).
split
(
'
\
n
'
)
if
p
.
strip
()
]
...
...
@@ -120,9 +115,11 @@ class ScriptBase(Eggs):
relative_paths
=
options
.
get
(
'relative-paths'
,
b_options
.
get
(
'relative-paths'
,
'false'
))
'relative-paths'
,
buildout
[
'buildout'
].
get
(
'relative-paths'
,
'false'
)
)
if
relative_paths
==
'true'
:
options
[
'buildout-directory'
]
=
b
_options
[
'directory'
]
options
[
'buildout-directory'
]
=
b
uildout
[
'buildout'
]
[
'directory'
]
self
.
_relative_paths
=
options
[
'buildout-directory'
]
else
:
self
.
_relative_paths
=
''
...
...
@@ -131,13 +128,12 @@ class ScriptBase(Eggs):
parse_entry_point
=
re
.
compile
(
'([^=]+)=(
\
w+(?:[.]
\
w+)*):(
\
w+(?:[.]
\
w+)*)$'
).
match
def
install
(
self
):
reqs
,
ws
=
self
.
working_set
()
options
=
self
.
options
scripts
=
options
.
get
(
'scripts'
)
if
scripts
or
scripts
is
None
or
options
.
get
(
'interpreter'
)
:
if
scripts
or
scripts
is
None
:
if
scripts
is
not
None
:
scripts
=
scripts
.
split
()
scripts
=
dict
([
...
...
@@ -161,31 +157,21 @@ class ScriptBase(Eggs):
name
=
dist
.
project_name
if
name
!=
'setuptools'
and
name
not
in
reqs
:
reqs
.
append
(
name
)
return
self
.
_install
(
reqs
,
ws
,
scripts
)
return
()
update
=
install
def
_install
(
self
,
reqs
,
ws
,
scripts
):
# Subclasses implement this.
raise
NotImplementedError
()
return
zc
.
buildout
.
easy_install
.
scripts
(
reqs
,
ws
,
options
[
'executable'
],
options
[
'bin-directory'
],
scripts
=
scripts
,
extra_paths
=
self
.
extra_paths
,
interpreter
=
options
.
get
(
'interpreter'
),
initialization
=
options
.
get
(
'initialization'
,
''
),
arguments
=
options
.
get
(
'arguments'
,
''
),
relative_paths
=
self
.
_relative_paths
,
)
class
Scripts
(
ScriptBase
):
def
_install
(
self
,
reqs
,
ws
,
scripts
):
options
=
self
.
options
return
zc
.
buildout
.
easy_install
.
scripts
(
reqs
,
ws
,
options
[
'executable'
],
options
[
'bin-directory'
],
scripts
=
scripts
,
extra_paths
=
self
.
extra_paths
,
interpreter
=
options
.
get
(
'interpreter'
),
initialization
=
options
.
get
(
'initialization'
,
''
),
arguments
=
options
.
get
(
'arguments'
,
''
),
relative_paths
=
self
.
_relative_paths
)
return
()
update
=
install
def
get_bool
(
options
,
name
,
default
=
False
):
value
=
options
.
get
(
name
)
...
...
zc.recipe.egg_/src/zc/recipe/egg/selecting-python.txt
View file @
6107db1f
...
...
@@ -25,7 +25,7 @@ We have a link server:
</body></html>
We have a sample buildout. Let's update it's configuration file to
install the demo package using Python 2.4.
install the demo package using Python 2.4.
>>> write(sample_buildout, 'buildout.cfg',
... """
...
...
@@ -35,7 +35,7 @@ install the demo package using Python 2.4.
... index = http://www.python.org/pypi/
...
... [python2.4]
... executable = %(python2
4
)s
... executable = %(python2
3
)s
...
... [demo]
... recipe = zc.recipe.egg
...
...
@@ -43,7 +43,7 @@ install the demo package using Python 2.4.
... find-links = %(server)s
... python = python2.4
... interpreter = py-demo
... """ % dict(server=link_server, python2
4
=other_executable))
... """ % dict(server=link_server, python2
3
=other_executable))
Now, if we run the buildout:
...
...
@@ -69,7 +69,7 @@ we'll get the Python 2.4 eggs for demo and demoneeded:
d setuptools-0.6-py2.4.egg
d setuptools-0.6-py2.5.egg
- zc.buildout-1.0-py2.5.egg
And the generated scripts invoke Python 2.4:
>>> import sys
...
...
@@ -81,8 +81,8 @@ And the generated scripts invoke Python 2.4:
>>> shebang = f.readline().strip()
>>> if shebang[:3] == '#!"' and shebang[-1] == '"':
... shebang = '#!'+shebang[3:-1]
>>>
assert shebang == '#!%s -S' % other_executable, (
... repr((shebang, '#!' + other_executable)))
>>>
shebang == '#!' + other_executable
True
>>> print f.read(), # doctest: +NORMALIZE_WHITESPACE
<BLANKLINE>
import sys
...
...
@@ -104,8 +104,8 @@ And the generated scripts invoke Python 2.4:
>>> shebang = f.readline().strip()
>>> if shebang[:3] == '#!"' and shebang[-1] == '"':
... shebang = '#!'+shebang[3:-1]
>>>
assert shebang == '#!%s -S' % other_executable, (
... repr((shebang, '#!' + other_executable)))
>>>
shebang == '#!' + other_executable
True
>>> print f.read(), # doctest: +NORMALIZE_WHITESPACE
<BLANKLINE>
import sys
...
...
zc.recipe.egg_/src/zc/recipe/egg/tests.py
View file @
6107db1f
...
...
@@ -36,7 +36,7 @@ def setUp(test):
def
setUpSelecting
(
test
):
zc
.
buildout
.
testselectingpython
.
setup
(
test
)
zc
.
buildout
.
testing
.
install_develop
(
'zc.recipe.egg'
,
test
)
def
test_suite
():
suite
=
unittest
.
TestSuite
((
doctest
.
DocFileSuite
(
...
...
@@ -67,7 +67,7 @@ def test_suite():
'setuptools-
\
S+
\
s+'
'zc.buildout-
\
S+
\
s*'
),
'__buildout_signature__ = sample- zc.recipe.egg-
\
n
'
),
'__buildout_signature__ = sample- zc.recipe.egg-'
),
(
re
.
compile
(
'executable = [
\
S ]+py
t
hon
\
S*
'
, re.I),
'
executable
=
python
'),
(re.compile('
find
-
links
=
http
:
//
localhost
:
\
d
+/
'),
...
...
@@ -89,7 +89,7 @@ def test_suite():
(re.compile('
extdemo
[.]
pyd
'), '
extdemo
.
so
')
]),
),
))
if sys.version_info[:2] == (2, 5):
...
...
@@ -115,7 +115,7 @@ def test_suite():
]),
),
)
return suite
if __name__ == '
__main__
':
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment