Skip to content

slapos-caddy
slapos-caddy
Commit 728de15e authored by Łukasz Nowak's avatar Łukasz Nowak
Browse files
Options

Merge branch 'erp5' 

Conflicts:
	setup.py
parents ed2729f5 32769afe
Show whitespace changes
Inline Side-by-side
Showing with 28 additions and 558 deletions
CHANGES.txt
View file @ 728de15e
0.2 (unreleased)
0.3 (unreleased)
================
* No changes yet.
0.2 (2011-05-30)
================
* Allow to pass zope_environment in erp5 entry point [Łukasz Nowak]
0.1 (2011-05-27)
================
......
setup.py
View file @ 728de15e
......@@ -2,7 +2,7 @@ from setuptools import setup, find_packages
import glob
import os
version = '0.2-dev'
version = '0.3-dev'
name = 'slapos.cookbook'
long_description = open("README.txt").read() + "\n" + \
open("CHANGES.txt").read() + "\n"
......@@ -28,11 +28,10 @@ setup(name=name,
install_requires=[
'PyXML', # for full blown python interpreter
'Zope2', # some recipes like to play with zope
'collective.recipe.template', # needed by template recipe
'lxml', # for full blown python interpreter
'netaddr', # to manipulate on IP addresses
'setuptools', # namespaces
'slapos.slap', # uses internally
'slapos.core', # uses internally
# 'slapos.toolbox', # needed for libcloud, cloudmgr, disabled for now
'xml_marshaller', # need to communication with slapgrid
'zc.buildout', # plays with buildout
......@@ -42,8 +41,6 @@ setup(name=name,
entry_points={
'zc.buildout': [
'bef_erp5 = slapos.recipe.bef_erp5:Recipe',
'build = slapos.recipe.build:Script',
'buildcmmi = slapos.recipe.build:Cmmi',
'download = slapos.recipe.download:Recipe',
'erp5 = slapos.recipe.erp5:Recipe',
'erp5testnode = slapos.recipe.erp5testnode:Recipe',
......@@ -58,7 +55,6 @@ setup(name=name,
'sheepdogtestbed = slapos.recipe.sheepdogtestbed:SheepDogTestBed',
'siptester = slapos.recipe.siptester:SipTesterRecipe',
'slaprunner = slapos.recipe.slaprunner:Recipe',
'template = slapos.recipe.template:Recipe',
'testnode = slapos.recipe.testnode:Recipe',
'vifib = slapos.recipe.vifib:Recipe',
'xwiki = slapos.recipe.xwiki:Recipe',
......
slapos/recipe/README.build.txt deleted 100644 → 0
View file @ ed2729f5
build
=====
Recipe to build the software.
Example buildout::
[buildout]
parts =
file
[zlib]
# Use standard configure, make, make install way
recipe = slapos.cookbook:build
url = http://prdownloads.sourceforge.net/libpng/zlib-1.2.5.tar.gz?download
md5sum = c735eab2d659a96e5a594c9e8541ad63
slapos_promisee =
directory:include
file:include/zconf.h
file:include/zlib.h
directory:lib
statlib:lib/libz.a
dynlib:lib/libz.so linked:libc.so.6 rpath:
dynlib:lib/libz.so.1 linked:libc.so.6 rpath:
dynlib:lib/libz.so.1.2.5 linked:libc.so.6
directory:lib/pkgconfig
file:lib/pkgconfig/zlib.pc
directory:share
directory:share/man
directory:share/man/man3
file:share/man/man3/zlib.3
[file]
recipe = slapos.cookbook:buildcmmi
url = ftp://ftp.astron.com/pub/file/file-5.04.tar.gz
md5sum = accade81ff1cc774904b47c72c8aeea0
environment =
CPPFLAGS=-I${zlib:location}/include
LDFLAGS=-L${zlib:location}/lib -Wl,-rpath -Wl,${zlib:location}/lib
slapos_promisee =
directory:bin
dynlib:bin/file linked:libz.so.1,libc.so.6,libmagic.so.1 rpath:${zlib:location}/lib,!/lib
directory:include
file:include/magic.h
directory:lib
statlib:lib/libmagic.a
statlib:lib/libmagic.la
dynlib:lib/libmagic.so linked:libz.so.1,libc.so.6 rpath:${zlib:location}/lib
dynlib:lib/libmagic.so.1 linked:libz.so.1,libc.so.6 rpath:${zlib:location}/lib
dynlib:lib/libmagic.so.1.0.0 linked:libz.so.1,libc.so.6 rpath:${zlib:location}/lib
directory:share
directory:share/man
directory:share/man/man1
file:share/man/man1/file.1
directory:share/man/man3
file:share/man/man3/libmagic.3
directory:share/man/man4
file:share/man/man4/magic.4
directory:share/man/man5
directory:share/misc
file:share/misc/magic.mgc
[somethingelse]
# default way with using script
recipe = slapos.cookbook:build
url_0 = http://host/path/file.tar.gz
md5sum = 9631070eac74f92a812d4785a84d1b4e
script =
import os
os.chdir(%(work_directory)s)
unpack(%(url_0), strip_path=True)
execute('make')
execute('make install DEST=%(location)s')
slapos_promisee =
...
TODO:
* add linking suport, buildout definition:
slapos_link = <relative/path> [optional-path
can be used as::
[file]
slapos_link =
bin/file
bin/file ${buildout:bin-directory}/bin/anotherfile
Which will link ${file:location}/bin/file to ${buildout:bin-directory}/bin/file
and ${file:location}/bin/file to ${buildout:bin-directory}/bin/anotherfile
slapos/recipe/README.template.txt deleted 100644 → 0
View file @ ed2729f5
template
========
Fully networked template recipe, reusing collective.recipe.template with
ability to download template over the network
Usage
-----
::
[buildout]
parts = template
[template]
recipe = slapos.cookbook:template
url = http://server/with/template
# optional md5sum
md5sum = 1234567890
output = ${buildout:directory}/result
All parameters except url and md5sum will be passed to
collective.recipe.template, so please visit
http://pypi.python.org/pypi/collective.recipe.template for full information.
slapos/recipe/build.py deleted 100644 → 0
View file @ ed2729f5
##############################################################################
#
# Copyright (c) 2010 Vifib SARL and Contributors. All Rights Reserved.
#
# WARNING: This program as such is intended to be used by professional
# programmers who take the whole responsibility of assessing all potential
# consequences resulting from its eventual inadequacies and bugs
# End users who are looking for a ready-to-use solution with commercial
# guarantees and support are strongly adviced to contract a Free Software
# Service Company
#
# This program is Free Software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 3
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
#
##############################################################################
import logging
import os
import setuptools
import shutil
import subprocess
import tempfile
import zc.buildout
def readElfAsDict(f):
"""Reads ELF information from file"""
popen = subprocess.Popen(['readelf', '-d', f],
stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
result = popen.communicate()[0]
if popen.returncode != 0:
raise AssertionError(result)
library_list = []
rpath_list = []
runpath_list = []
for l in result.split('\n'):
if '(NEEDED)' in l:
library_list.append(l.split(':')[1].strip(' []'))
elif '(RPATH)' in l:
rpath_list = [q.rstrip('/') for q in l.split(':',1)[1].strip(' []').split(':')]
elif '(RUNPATH)' in l:
runpath_list = [q.rstrip('/') for q in l.split(':',1)[1].strip(' []').split(':')]
if len(runpath_list) == 0:
runpath_list = rpath_list
elif len(rpath_list) != 0 and runpath_list != rpath_list:
raise ValueError('RPATH and RUNPATH are different.')
return dict(
library_list=sorted(library_list),
runpath_list=sorted(runpath_list)
)
def call(*args, **kwargs):
"""Subprocess call with closed file descriptors and stdin"""
kwargs.update(
stdin=subprocess.PIPE,
close_fds=True)
popen = subprocess.Popen(*args, **kwargs)
popen.stdin.flush()
popen.stdin.close()
popen.stdin = None
popen.communicate()
if popen.returncode != 0:
raise subprocess.CalledProcessError(popen.returncode, ' '.join(args[0]))
def calls(call_string, **kwargs):
"""Subprocesser caller which allows to pass arguments as string"""
call(call_string.split(), **kwargs)
def guessworkdir(path):
if len(os.listdir(path)) == 1:
return os.path.join(path, os.listdir(path)[0])
return path
class Script:
"""Free script building system"""
def _checkPromisee(self, location):
promisee_problem_list = []
a = promisee_problem_list.append
for promisee in self.options['slapos_promisee'].split('\n'):
promisee = promisee.strip()
if not promisee:
continue
if promisee.startswith('file:') or promisee.startswith('statlib'):
s, path = promisee.split(':')
if not os.path.exists(os.path.join(location, path)):
a('File promisee not met for %r' % path)
elif promisee.startswith('directory'):
s, path = promisee.split(':')
if not os.path.isdir(os.path.join(location, path)):
a('Directory promisee not met for %r' %
path)
elif promisee.startswith('dynlib:'):
if 'linked:' not in promisee:
raise zc.buildout.UserError('dynlib promisee requires \'linked:\' '
'parameter.')
if 'rpath:' not in promisee:
rpath_list = []
for promisee_part in promisee.split():
if promisee_part.startswith('dynlib:'):
s, path = promisee_part.split(':')
elif promisee_part.startswith('linked:'):
s, link_list = promisee_part.split(':')
link_list = link_list.split(',')
elif promisee_part.startswith('rpath:'):
s, rpath_list = promisee_part.split(':')
if rpath_list:
r = rpath_list
rpath_list = []
for q in r.split(','):
if q.startswith('!'):
q = q.replace('!', location)
rpath_list.append(q)
else:
rpath_list = []
if not os.path.exists(os.path.join(location, path)):
a('Dynlib promisee file not met %r' % promisee)
else:
elf_dict = readElfAsDict(os.path.join(location, path))
if sorted(link_list) != sorted(elf_dict['library_list']):
a('Promisee library list not met (wanted: %r, found: %r)'%(
link_list, elf_dict['library_list']))
if sorted(rpath_list) != sorted(elf_dict['runpath_list']):
a('Promisee rpath list not met (wanted: %r, found: %r)'%(
rpath_list, elf_dict['runpath_list']))
else:
raise zc.buildout.UserError('Unknown promisee %r' % promisee)
if len(promisee_problem_list):
raise zc.buildout.UserError('Promisee not met, found issues:\n %s' %
' '.join([q+'\n' for q in promisee_problem_list]))
def download(self, url, md5sum):
download = zc.buildout.download.Download(self.buildout['buildout'],
hash_name=True)
path, is_temp = download(url, md5sum=self.options.get('md5sum'))
return path
def extract(self, path):
extract_dir = tempfile.mkdtemp(self.name)
self.logger.debug('Created working directory %r' % extract_dir)
setuptools.archive_util.unpack_archive(path, extract_dir)
self.cleanup_dir_list.append(extract_dir)
return extract_dir
script = 'raise NotImplementedError'
def __init__(self, buildout, name, options):
self.cleanup_dir_list = []
self.options = options
self.buildout = buildout
self.name = name
self.logger = logging.getLogger('SlapOS build of %s' % self.name)
self.options.setdefault('location',
os.path.join(buildout['buildout']['parts-directory'], self.name))
# cleanup some variables
for k in ['location', 'url', 'md5sum']:
self.options[k] = self.options.get(k, '').strip()
self.options['script'] = self.options.get('script', self.script) % self.options
def getEnvironment(self):
# prepare cool dictionary
wanted_env = {}
for line in self.options.get('environment', '').splitlines():
line = line.strip()
if not line:
continue
if not '=' in line:
raise zc.buildout.UserError('Line %r in environment is incorrect' % line)
key, value = line.split('=')
key = key.strip()
value = value.strip()
if key in wanted_env:
raise zc.buildout.UserError('Key %r is repeated' % key)
wanted_env[key] = value
env = {}
for k,v in os.environ.iteritems():
change = wanted_env.pop(k, None)
if change is not None:
env[k] = change % os.environ
self.logger.info('Environment %r setup to %r' % (k, env[k]))
else:
env[k] =v
for k,v in wanted_env.iteritems():
self.logger.info('Environment %r added with %r' % (k, v))
env[k] = v
return env
def install(self):
try:
env = self.getEnvironment()
exec self.options['script']
try:
self._checkPromisee(self.options['location'])
except Exception:
if os.path.exists(self.options['location']):
self.logger.info('Removing location %r because of error' % self.options['location'])
shutil.rmtree(self.options['location'])
raise
finally:
for d in self.cleanup_dir_list:
if os.path.exists(d):
self.logger.debug('Cleanup directory %r' % d)
shutil.rmtree(d)
return [self.options['location']]
def update(self):
pass
class Cmmi(Script):
"""Simple configure-make-make-insall compatible with hexagonit.recipe.cmmi
Compatibility on parameter level, without bug-to-bug, hack-to-hack"""
script = """
extract_dir = self.extract(self.download(self.options['url'], self.options.get('md5sum')))
workdir = guessworkdir(extract_dir)
configure_command = ["./configure", "--prefix=%(location)s"]
configure_command.extend(%(configure-options)r.split())
self.logger.info('Configuring with: %%s' %% configure_command)
call(configure_command, cwd=workdir, env=env)
self.logger.info('Building')
call("make", cwd=workdir, env=env)
self.logger.info('Installing')
call(["make", "install"], cwd=workdir, env=env)
"""
def __init__(self, buildout, name, options):
options['configure-options'] = ' '.join(options.get('configure-options', '').strip().splitlines())
Script.__init__(self, buildout, name, options)
slapos/recipe/erp5/__init__.py
View file @ 728de15e
......@@ -617,7 +617,19 @@ class Recipe(BaseSlapRecipe):
def installZope(self, ip, port, name, zodb_configuration_string,
with_timerservice=False, tidstorage_config=None, thread_amount=1,
with_deadlockdebugger=True):
with_deadlockdebugger=True, zope_environment=None):
default_zope_environment = dict(
TMP=self.tmp_directory,
TMPDIR=self.tmp_directory,
HOME=self.tmp_directory,
PATH=self.bin_directory
)
if zope_environment is None:
zope_environment = default_zope_environment.copy()
else:
for envk, envv in default_zope_environment.iteritems():
if envk not in zope_environment:
zope_environment[envk] = envv
# Create zope configuration file
zope_config = dict(
products=self.options['products'],
......@@ -649,8 +661,10 @@ class Recipe(BaseSlapRecipe):
self.erp5_directory, 'Products'))
zope_config['products'] = '\n'.join(prefixed_products)
zope_config['address'] = '%s:%s' % (ip, port)
zope_config['tmp_directory'] = self.tmp_directory
zope_config['path'] = self.bin_directory
zope_environment_list = []
for envk, envv in zope_environment.iteritems():
zope_environment_list.append('%s %s' % (envk, envv))
zope_config['environment'] = "\n".join(zope_environment_list)
zope_wrapper_template_location = self.getTemplateFilename('zope.conf.in')
zope_conf_content = self.substituteTemplate(
......
slapos/recipe/erp5/template/zope.conf.in
View file @ 728de15e
......@@ -9,10 +9,7 @@ instancehome $INSTANCE
# Environment override
<environment>
TMP %(tmp_directory)s
TMPDIR %(tmp_directory)s
HOME %(tmp_directory)s
PATH %(path)s
%(environment)s
</environment>
# No need to debug
......
slapos/recipe/template.py deleted 100644 → 0
View file @ ed2729f5
##############################################################################
#
# Copyright (c) 2010 Vifib SARL and Contributors. All Rights Reserved.
#
# WARNING: This program as such is intended to be used by professional
# programmers who take the whole responsibility of assessing all potential
# consequences resulting from its eventual inadequacies and bugs
# End users who are looking for a ready-to-use solution with commercial
# guarantees and support are strongly adviced to contract a Free Software
# Service Company
#
# This program is Free Software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 3
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
#
##############################################################################
import collective.recipe.template
import zc.buildout
class Recipe(collective.recipe.template.Recipe):
def __init__(self, buildout, name, options):
download = zc.buildout.download.Download(buildout['buildout'],
hash_name=True)
path, is_temp = download(options.pop('url'),
md5sum=options.get('md5sum'))
options['input'] = path
collective.recipe.template.Recipe.__init__(self, buildout, name, options)
software/erp5/software.cfg
View file @ 728de15e
......@@ -37,153 +37,3 @@ url = ${:_profile_base_location_}/instance.cfg
md5sum = 1b29b27aedcc7fa5f30f1053e8eab13f
output = ${buildout:directory}/template.cfg
mode = 0644
# Release stabilisation
[products-deps]
# Recipe minitage.recipe.fetch is disabled, as it uses PATH variable, but it
# is not possible to change its environment to use localy delivered subversion
# nor git. plone.recipe.command can do same job, but it is controllable which
# binary will be used
recipe = plone.recipe.command
svn_param =--trust-server-cert --non-interactive --quiet
location = ${buildout:parts-directory}/${:_buildout_section_name_}
stop-on-error = true
command =
${subversion:location}/bin/svn checkout ${:svn_param} -r 98997 https://svn.plone.org/svn/collective/ExtFile/trunk ${:location}/ExtFile &&
${git:location}/bin/git clone --quiet git://git.hforge.org/Localizer.git ${:location}/Localizer && cd ${:location}/Localizer && ${git:location}/bin/git reset --quiet --hard dacb6ba0ae559cd9bdb8822812d24a12a21e9e37
update-command =
[erp5]
# Recipe zerokspot.recipe.git is disabled, as is not possible to change its
# environment to use localy delivered git.
# plone.recipe.command can do same job, but it is controllable which binary
# will be used
recipe = plone.recipe.command
location = ${buildout:parts-directory}/${:_buildout_section_name_}
stop-on-error = true
repository = http://git.erp5.org/repos/erp5.git
branch = master
command = ${git:location}/bin/git clone --quiet -b ${:branch} ${:repository} ${:location} && cd ${:location} && ${git:location}/bin/git reset --quiet --hard 0eba9830dae07dbf5319f218989b79ddd5eb11b1
update-command =
[versions]
MySQL-python = 1.2.3
Paste = 1.7.5.1
PasteScript = 1.7.3
Products.CMFActionIcons = 2.1.3
Products.CMFCalendar = 2.2.2
Products.CMFCore = 2.2.4
Products.CMFDefault = 2.2.2
Products.CMFTopic = 2.2.1
Products.CMFUid = 2.2.1
Products.DCWorkflowGraph = 0.4nxd001
Products.ExternalEditor = 1.1.0
Products.GenericSetup = 1.6.3
Products.MimetypesRegistry = 2.0.2
Products.PluggableAuthService = 1.7.4
Products.PluginRegistry = 1.3b1
Products.TIDStorage = 5.4.7.dev-r45842
Products.Zelenium = 1.0.3
StructuredText = 2.11.1
cElementTree = 1.0.5-20051216
chardet = 1.0.1
cloudooo = 1.2.3
cloudooo.handler.ffmpeg = 0.1
cloudooo.handler.imagemagick = 0.1
cloudooo.handler.ooo = 0.2
cloudooo.handler.pdf = 0.1
csp-eventlet = 0.6.0
elementtree = 1.2.7-20070827-preview
erp5.conflictresolver = 0.3
erp5.recipe.cmmiforcei686 = 0.1.1
erp5diff = 0.8.1.3
eventlet = 0.9.15
feedparser = 5.0.1
five.localsitemanager = 2.0.5
greenlet = 0.3.1
hexagonit.recipe.cmmi = 1.5.0
hexagonit.recipe.download = 1.5.0
ipdb = 0.3
mr.developer = 1.17
ordereddict = 1.1
paramiko = 1.7.7.1
plone.recipe.command = 1.1
ply = 3.4
psutil = 0.2.1
pycrypto = 2.3
python-ldap = 2.3.13
python-memcached = 1.47
restkit = 3.2.3
rtjp-eventlet = 0.3.2
slapos.cookbook = 0.1
threadframe = 0.2
timerserver = 2.0.2
urlnorm = 1.1.2
uuid = 1.30
xupdate-processor = 0.4
# Required by:
# PasteScript==1.7.3
# cloudooo==1.2.3
PasteDeploy = 1.5.0
# Required by:
# cloudooo==1.2.3
WSGIUtils = 0.7
# Required by:
# cloudooo==1.2.3
argparse = 1.2.1
# Required by:
# slapos.cookbook==0.1
collective.recipe.template = 1.8
# Required by:
# SOAPpy==0.12.0nxd001
fpconst = 0.7.2
# Required by:
# ipdb==0.3
ipython = 0.10.2
# Required by:
# slapos.cookbook==0.1
netaddr = 0.7.5
# Required by:
# cloudooo==1.2.3
python-magic = 0.4.0.1
# Required by:
# Products.CMFActionIcons==2.1.3
# Products.CMFCalendar==2.2.2
# Products.CMFCore==2.2.4
# Products.CMFDefault==2.2.2
# Products.CMFTopic==2.2.1
# Products.CMFUid==2.2.1
# Products.DCWorkflow==2.2.3nxd002
# Products.DCWorkflowGraph==0.4nxd001
# Products.ExternalEditor==1.1.0
# Products.GenericSetup==1.6.3
# Products.MimetypesRegistry==2.0.2
# Products.PluggableAuthService==1.7.4
# Products.PluginRegistry==1.3b1
# Products.TIDStorage==5.4.7.dev-r45842
# Products.Zelenium==1.0.3
# Zope2==2.12.18
# five.localsitemanager==2.0.5
# mr.developer==1.17
# python-ldap==2.3.13
# zc.buildout==1.5.3-dev-SlapOS-001
# zope.deprecation==3.4.0
# zope.structuredtext==3.4.0
setuptools = 0.6c12dev-r88795
# Required by:
# slapos.cookbook==0.1
slapos.slap = 1.2.dev-r4679
# Required by:
# slapos.cookbook==0.1
xml-marshaller = 0.9.7
software/erp5testnode/software.cfg
View file @ 728de15e
......@@ -50,7 +50,7 @@ eggs =
slapos.toolbox
[template]
recipe = slapos.cookbook:template
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/testnode-instance.cfg
md5sum = 0b44b72c9e21d760f7e27a89e9d10187
output = ${buildout:directory}/template.cfg
......
stack/erp5.cfg
View file @ 728de15e
......@@ -120,7 +120,7 @@ section = python2.7
[template]
# Default template for erp5 instance.
recipe = slapos.cookbook:template
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance.cfg
md5sum = 16d09f1964101bbe128a81c7ffcf996e
output = ${buildout:directory}/template.cfg
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7