Skip to content

S
slapos.buildout
Commit 4f7b87de authored by Jim Fulton's avatar Jim Fulton
Browse files
Options

merged origin

parents 9dcf920f 8df1e8db
Show whitespace changes
Inline Side-by-side
Showing with 298 additions and 65 deletions
CHANGES.rst
View file @ 4f7b87de
Change History
**************
Unreleased
==========
- Open files for ``exec()`` in universal newlines mode. See
https://github.com/buildout/buildout/issues/130
- Add ``BUILDOUT_HOME`` as an alternate way to control how the user default
configuration is found.
- Close various files when finished writing to them. This avoids
ResourceWarnings on Python 3, and better supports doctests under PyPy.
2.2.1 (2013-09-05)
==================
- ``distutils`` scripts: correct order of operations on ``from ... import``
lines (see https://github.com/buildout/buildout/issues/134).
- Add an ``--allow-site-packges`` option to ``bootstrap.py``, defaulting
to False. If the value is false, strip any "site packages" (as defined by
the ``site`` module) from ``sys.path`` before attempting to import
``setuptools`` / ``pkg_resources``.
- Updated the URL used to fetch ``ez_setup.py`` to the official, non-version-
pinned version.
2.2.0 (2013-07-05)
==================
......
Makefile
View file @ 4f7b87de
......@@ -16,7 +16,7 @@ ifeq ($(PYTHON_VER),3.3)
endif
PYTHON_ARCHIVE ?= Python-$(PYTHON_MINOR)
PYTHON_DOWNLOAD = http://www.python.org/ftp/python/$(PYTHON_MINOR)/$(PYTHON_ARCHIVE).tgz
PYTHON_DOWNLOAD = https://www.python.org/ftp/python/$(PYTHON_MINOR)/$(PYTHON_ARCHIVE).tgz
PYTHON_EXE = python$(PYTHON_VER)
.PHONY: all build test
......@@ -28,12 +28,12 @@ $(PYTHON_PATH):
@echo "Installing Python"
mkdir -p $(PYTHON_PATH)
cd $(PYTHON_PATH) && \
curl --progress-bar $(PYTHON_DOWNLOAD) | tar -zx
curl --progress-bar --location $(PYTHON_DOWNLOAD) | tar -zx
ifeq ($(PYTHON_VER),2.6)
cd $(PYTHON_PATH) && \
curl --progress-bar https://raw.github.com/collective/buildout.python/master/src/issue12012-sslv2-py26.txt > ssl.txt
curl --progress-bar https://raw.github.com/collective/buildout.python/ad45adb78bfa37542d62a394392d5146fce5af34/src/issue12012-sslv2-py26.patch > ssl.patch
cd $(PYTHON_PATH)/$(PYTHON_ARCHIVE) && \
patch -p0 < ../ssl.txt
patch -p0 < ../ssl.patch
endif
cd $(PYTHON_PATH)/$(PYTHON_ARCHIVE) && \
./configure --prefix $(PYTHON_PATH) >/dev/null 2>&1 && \
......
README.rst
View file @ 4f7b87de
......@@ -25,7 +25,7 @@ Buildout is a project designed to solve 2 problems:
Buildout might be confused with build tools like make or ant, but
it is a little higher level and might invoke systems like make or
ant to get it's work done.
ant to get its work done.
Buildout might be confused with systems like puppet or chef, but it
is more application focused. Systems like puppet or chef might
......
bootstrap/bootstrap.py
View file @ 4f7b87de
......@@ -56,6 +56,9 @@ parser.add_option("-c", "--config-file",
"file to be used."))
parser.add_option("-f", "--find-links",
help=("Specify a URL to search for buildout releases"))
parser.add_option("--allow-site-packages",
action="store_true", default=False,
help=("Let bootstrap.py use existing site packages"))
options, args = parser.parse_args()
......@@ -63,28 +66,36 @@ options, args = parser.parse_args()
######################################################################
# load/install setuptools
to_reload = False
try:
import pkg_resources
if options.allow_site_packages:
import setuptools
except ImportError:
ez = {}
try:
import pkg_resources
from urllib.request import urlopen
except ImportError:
except ImportError:
from urllib2 import urlopen
exec(urlopen('https://bootstrap.pypa.io/ez_setup.py').read(), ez)
setup_args = dict(to_dir=tmpeggs, download_delay=0)
ez['use_setuptools'](**setup_args)
if to_reload:
reload(pkg_resources)
import pkg_resources
# This does not (always?) update the default working set. We will
# do it.
for path in sys.path:
ez = {}
exec(urlopen('https://bootstrap.pypa.io/ez_setup.py').read(), ez)
if not options.allow_site_packages:
# ez_setup imports site, which adds site packages
# this will remove them from the path to ensure that incompatible versions
# of setuptools are not in the path
import site
# inside a virtualenv, there is no 'getsitepackages'.
# We can't remove these reliably
if hasattr(site, 'getsitepackages'):
for sitepackage_path in site.getsitepackages():
sys.path[:] = [x for x in sys.path if sitepackage_path not in x]
setup_args = dict(to_dir=tmpeggs, download_delay=0)
ez['use_setuptools'](**setup_args)
import setuptools
import pkg_resources
# This does not (always?) update the default working set. We will
# do it.
for path in sys.path:
if path not in pkg_resources.working_set.entries:
pkg_resources.working_set.add_entry(path)
......
setup.py
View file @ 4f7b87de
......@@ -12,7 +12,7 @@
#
##############################################################################
name = "zc.buildout"
version = "2.2.1dev"
version = "2.2.2dev"
import os
from setuptools import setup
......
src/zc/buildout/buildout.py
View file @ 4f7b87de
......@@ -208,8 +208,12 @@ class Buildout(DictMixin):
# load user defaults, which override defaults
if user_defaults:
user_config = os.path.join(os.path.expanduser('~'),
'.buildout', 'default.cfg')
if os.environ.get('BUILDOUT_HOME'):
buildout_home = os.environ['BUILDOUT_HOME']
else:
buildout_home = os.path.join(
os.path.expanduser('~'), '.buildout')
user_config = os.path.join(buildout_home, 'default.cfg')
if os.path.exists(user_config):
_update(data, _open(os.path.dirname(user_config), user_config,
[], data['buildout'].copy(), override,
......
src/zc/buildout/buildout.txt
View file @ 4f7b87de
......@@ -1725,7 +1725,54 @@ user defaults:
op5 b3base 5
recipe recipes:debug
If the environment variable BUILDOUT_HOME is non-empty, that is used to
locate default.cfg instead of looking in ~/.buildout/. Let's set up a
configuration file in an alternate directory and verify that we get the
appropriate set of defaults:
>>> alterhome = tmpdir('alterhome')
>>> write(alterhome, 'default.cfg',
... """
... [debug]
... op1 = 1'
... op7 = 7'
... op8 = eight!
... """)
>>> os.environ['BUILDOUT_HOME'] = alterhome
>>> print_(system(buildout), end='')
Develop: '/sample-buildout/recipes'
Uninstalling debug.
Installing debug.
name base
op buildout
op1 b1 1
op2 b2 2
op3 b2 3
op4 b3 4
op5 b3base 5
op7 7'
op8 eight!
recipe recipes:debug
The -U argument still suppresses reading of the default.cfg file from
BUILDOUT_HOME:
>>> print_(system(buildout + ' -U'), end='')
Develop: '/sample-buildout/recipes'
Uninstalling debug.
Installing debug.
name base
op buildout
op1 b1 1
op2 b2 2
op3 b2 3
op4 b3 4
op5 b3base 5
recipe recipes:debug
>>> os.environ['HOME'] = old_home
>>> del os.environ['BUILDOUT_HOME']
Log level
---------
......
src/zc/buildout/easy_install.py
View file @ 4f7b87de
......@@ -19,6 +19,7 @@ installed.
"""
import distutils.errors
import errno
import glob
import logging
import os
......@@ -868,7 +869,8 @@ def develop(setup, dest,
os.rename(setup_cfg+'-develop-aside', setup_cfg)
undo.append(restore_old_setup)
else:
open(setup_cfg, 'w')
f = open(setup_cfg, 'w')
f.close()
undo.append(lambda: os.remove(setup_cfg))
setuptools.command.setopt.edit_config(
setup_cfg, dict(build_ext=build_ext))
......@@ -1107,7 +1109,7 @@ def _distutils_script(path, dest, script_content, initialization, rsetup):
for line_number, line in enumerate(lines):
if not 'import' in line:
continue
if not line.startswith('import') or line.startswith('from'):
if not (line.startswith('import') or line.startswith('from')):
continue
if '__future__' in line:
continue
......@@ -1129,12 +1131,21 @@ def _distutils_script(path, dest, script_content, initialization, rsetup):
)
return _create_script(contents, dest)
def _file_changed(filename, old_contents, mode='r'):
try:
with open(filename, mode) as f:
return f.read() != old_contents
except EnvironmentError as e:
if e.errno == errno.ENOENT:
return True
else:
raise
def _create_script(contents, dest):
generated = []
script = dest
changed = not (os.path.exists(dest) and open(dest).read() == contents)
changed = _file_changed(dest, contents)
if is_win32:
# generate exe file and give the script a magic name:
......@@ -1147,15 +1158,16 @@ def _create_script(contents, dest):
except AttributeError:
# fall back for compatibility with older Distribute versions
new_data = pkg_resources.resource_string('setuptools', 'cli.exe')
if (not os.path.exists(win32_exe) or
(open(win32_exe, 'rb').read() != new_data)
):
if _file_changed(win32_exe, new_data, 'rb'):
# Only write it if it's different.
open(win32_exe, 'wb').write(new_data)
with open(win32_exe, 'wb') as f:
f.write(new_data)
generated.append(win32_exe)
if changed:
open(dest, 'w').write(contents)
with open(dest, 'w') as f:
f.write(contents)
logger.info(
"Generated script %r.",
# Normalize for windows
......@@ -1218,18 +1230,20 @@ def _pyscript(path, dest, rsetup, initialization=''):
relative_paths_setup = rsetup,
initialization=initialization,
)
changed = not (os.path.exists(dest) and open(dest).read() == contents)
changed = _file_changed(dest, contents)
if is_win32:
# generate exe file and give the script a magic name:
exe = script + '.exe'
open(exe, 'wb').write(
with open(exe, 'wb') as f:
f.write(
pkg_resources.resource_string('setuptools', 'cli.exe')
)
generated.append(exe)
if changed:
open(dest, 'w').write(contents)
with open(dest, 'w') as f:
f.write(contents)
try:
os.chmod(dest, _execute_permission())
except (AttributeError, os.error):
......@@ -1268,9 +1282,8 @@ if len(sys.argv) > 1:
sys.argv[:] = _args
__file__ = _args[0]
del _options, _args
__file__f = open(__file__)
with open(__file__, 'U') as __file__f:
exec(compile(__file__f.read(), __file__, "exec"))
__file__f.close(); del __file__f
if _interactive:
del _interactive
......@@ -1289,7 +1302,8 @@ __file__ = %(__file__)r
os.chdir(%(setupdir)r)
sys.argv[0] = %(setup)r
exec(compile(open(%(setup)r).read(), %(setup)r, 'exec'))
with open(%(setup)r, 'U') as f:
exec(compile(f.read(), %(setup)r, 'exec'))
"""
......
src/zc/buildout/easy_install.txt
View file @ 4f7b87de
......@@ -614,7 +614,7 @@ run other scripts with the path set on the working set:
The py script simply runs the Python interactive interpreter with
the path set:
>>> cat(bin, 'py') # doctest: +NORMALIZE_WHITESPACE
>>> cat(bin, 'py') # doctest: +NORMALIZE_WHITESPACE +REPORT_NDIFF
#!/usr/local/bin/python2.7
<BLANKLINE>
import sys
......@@ -643,9 +643,8 @@ the path set:
sys.argv[:] = _args
__file__ = _args[0]
del _options, _args
__file__f = open(__file__)
with open(__file__, 'U') as __file__f:
exec(compile(__file__f.read(), __file__, "exec"))
__file__f.close(); del __file__f
<BLANKLINE>
if _interactive:
del _interactive
......@@ -872,7 +871,7 @@ Of course, running the script works:
We specified an interpreter and its paths are adjusted too:
>>> cat(bo, 'bin', 'py') # doctest: +NORMALIZE_WHITESPACE
>>> cat(bo, 'bin', 'py') # doctest: +NORMALIZE_WHITESPACE +REPORT_NDIFF
#!/usr/local/bin/python2.7
<BLANKLINE>
import os
......@@ -909,9 +908,8 @@ We specified an interpreter and its paths are adjusted too:
sys.argv[:] = _args
__file__ = _args[0]
del _options, _args
__file__f = open(__file__)
with open(__file__, 'U') as __file__f:
exec(compile(__file__f.read(), __file__, "exec"))
__file__f.close(); del __file__f
<BLANKLINE>
if _interactive:
del _interactive
......
src/zc/buildout/testing.py
View file @ 4f7b87de
......@@ -56,7 +56,8 @@ def cat(dir, *names):
and os.path.exists(path+'-script.py')
):
path = path+'-script.py'
print_(open(path).read(), end='')
with open(path) as f:
print_(f.read(), end='')
def ls(dir, *subs):
if subs:
......@@ -240,9 +241,8 @@ def buildoutSetUp(test):
os.chdir(sample)
# Create a basic buildout.cfg to avoid a warning from buildout:
open('buildout.cfg', 'w').write(
"[buildout]\nparts =\n"
)
with open('buildout.cfg', 'w') as f:
f.write("[buildout]\nparts =\n")
# Use the buildout bootstrap command to create a buildout
zc.buildout.buildout.Buildout(
......@@ -375,7 +375,8 @@ class Handler(BaseHTTPRequestHandler):
self.send_header('Content-Length', str(len(out)))
self.send_header('Content-Type', 'text/html')
else:
out = open(path, 'rb').read()
with open(path, 'rb') as f:
out = f.read()
self.send_header('Content-Length', len(out))
if path.endswith('.egg'):
self.send_header('Content-Type', 'application/zip')
......@@ -471,8 +472,8 @@ def install(project, destination):
shutil.copyfile(dist.location, destination)
else:
# copy link
open(os.path.join(destination, project+'.egg-link'), 'w'
).write(dist.location)
with open(os.path.join(destination, project+'.egg-link'), 'w') as f:
f.write(dist.location)
def install_develop(project, destination):
if not isinstance(destination, str):
......@@ -481,8 +482,8 @@ def install_develop(project, destination):
dist = pkg_resources.working_set.find(
pkg_resources.Requirement.parse(project))
open(os.path.join(destination, project+'.egg-link'), 'w'
).write(dist.location)
with open(os.path.join(destination, project+'.egg-link'), 'w') as f:
f.write(dist.location)
def _normalize_path(match):
path = match.group(1)
......
src/zc/buildout/tests.py
View file @ 4f7b87de
......@@ -817,7 +817,7 @@ On the other hand, if we have a regular egg, rather than a develop egg:
>>> ls('eggs') # doctest: +ELLIPSIS
- foox-0.0.0-py2.4.egg
- setuptools.eggpyN.N.egg
d setuptools.eggpyN.N.egg
...
We do not get a warning, but we do get setuptools included in the working set:
......@@ -2747,6 +2747,77 @@ def test_constrained_requirement():
... print_('failed', o, c, g, '!=', e)
"""
def test_distutils_scripts_using_import_are_properly_parsed():
"""
zc.buildout.easy_install._distutils_script(path, dest, script_content, initialization, rsetup):
Creates a script for a distutils based project. In this example for a
hypothetical code quality checker called 'pyflint' that uses an import
statement to import its code.
>>> pyflint_script = '''#!/path/to/bin/python
... import pyflint.do_something
... pyflint.do_something()
... '''
>>> import sys
>>> original_executable = sys.executable
>>> sys.executable = 'python'
>>> from zc.buildout.easy_install import _distutils_script
>>> _distutils_script('\\'/path/test/\\'', 'bin/pyflint', pyflint_script, '', '')
['bin/pyflint']
>>> cat('bin/pyflint')
#!python
<BLANKLINE>
<BLANKLINE>
import sys
sys.path[0:0] = [
'/path/test/',
]
<BLANKLINE>
<BLANKLINE>
import pyflint.do_something
pyflint.do_something()
>>> sys.executable = original_executable
"""
def test_distutils_scripts_using_from_are_properly_parsed():
"""
zc.buildout.easy_install._distutils_script(path, dest, script_content, initialization, rsetup):
Creates a script for a distutils based project. In this example for a
hypothetical code quality checker called 'pyflint' that uses a from
statement to import its code.
>>> pyflint_script = '''#!/path/to/bin/python
... from pyflint import do_something
... do_something()
... '''
>>> import sys
>>> original_executable = sys.executable
>>> sys.executable = 'python'
>>> from zc.buildout.easy_install import _distutils_script
>>> _distutils_script('\\'/path/test/\\'', 'bin/pyflint', pyflint_script, '', '')
['bin/pyflint']
>>> cat('bin/pyflint')
#!python
<BLANKLINE>
<BLANKLINE>
import sys
sys.path[0:0] = [
'/path/test/',
]
<BLANKLINE>
<BLANKLINE>
from pyflint import do_something
do_something()
>>> sys.executable = original_executable
"""
def want_new_zcrecipeegg():
"""
>>> write('buildout.cfg',
......@@ -3421,6 +3492,7 @@ def test_suite():
r'We have a develop egg: \1 V'),
(re.compile('Picked: setuptools = \S+'),
'Picked: setuptools = V'),
(re.compile('[-d] setuptools'), '- setuptools'),
(re.compile(r'\\[\\]?'), '/'),
(re.compile(
'-q develop -mxN -d "/sample-buildout/develop-eggs'),
......
zc.recipe.egg_/CHANGES.txt
View file @ 4f7b87de
Change History
**************
2.0.1 (2013-09-05)
==================
- Accomodate ``zc.buildout`` switch to post-merge ``setuptools``.
2.0.0 (2013-04-02)
==================
- Enabled 'prefer-final' option by default.
2.0.0a3 (2012-11-19)
====================
- Added support for Python 3.2 / 3.3.
- Added 'MANIFEST.in'.
- Support non-entry-point-based scripts.
- Honor exit codes from scripts (https://bugs.launchpad.net/bugs/697913).
2.0.0a2 (2012-05-03)
====================
- Always unzip installed eggs.
- Switched from using 'setuptools' to 'distribute'.
- Removed multi-python support.
1.3.2 (2010-08-23)
==================
- Bugfix for the change introduced in 1.3.1.
1.3.1 (2010-08-23)
==================
- Support recipes that are using zc.recipe.egg by passing in a dict, rather
than a zc.buildout.buildout.Options object as was expected/tested.
1.3.0 (2010-08-23)
==================
- Small further refactorings past 1.2.3b1 to be compatible with
zc.buildout 1.5.0.
1.2.3b1 (2010-04-29)
====================
- Refactored to be used with z3c.recipe.scripts and zc.buildout 1.5.0.
No new user-visible features.
1.2.2 (2009-03-18)
==================
......
zc.recipe.egg_/setup.py
View file @ 4f7b87de
......@@ -14,7 +14,7 @@
"""Setup for zc.recipe.egg package
"""
version = '2.0.1dev'
version = '2.0.1'
import os
from setuptools import setup, find_packages
......@@ -52,6 +52,13 @@ setup(
'Framework :: Buildout',
'Intended Audience :: Developers',
'License :: OSI Approved :: Zope Public License',
'Programming Language :: Python',
'Programming Language :: Python :: 2',
'Programming Language :: Python :: 2.6',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.2',
'Programming Language :: Python :: 3.3',
'Topic :: Software Development :: Build Tools',
'Topic :: Software Development :: Libraries :: Python Modules',
],
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7