Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
S
slapos.recipe.cmmi
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
Analytics
Analytics
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Commits
Issue Boards
Open sidebar
Dmitry Blinov
slapos.recipe.cmmi
Commits
50958d2e
Commit
50958d2e
authored
Feb 21, 2013
by
Jondy Zhao
Committed by
Cédric de Saint Martin
Apr 12, 2013
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Add support for multiple platforms.
parent
f10c38e3
Changes
3
Show whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
813 additions
and
11 deletions
+813
-11
slapos/recipe/cmmi/README.txt
slapos/recipe/cmmi/README.txt
+549
-3
slapos/recipe/cmmi/__init__.py
slapos/recipe/cmmi/__init__.py
+158
-8
slapos/recipe/cmmi/tests.py
slapos/recipe/cmmi/tests.py
+106
-0
No files found.
slapos/recipe/cmmi/README.txt
View file @
50958d2e
...
@@ -89,10 +89,13 @@ Supported options
...
@@ -89,10 +89,13 @@ Supported options
``configure`` script. The format of the options is::
``configure`` script. The format of the options is::
/path/to/the/module.py:name_of_callable
/path/to/the/module.py:name_of_callable
url:name_of_callable
url#md5sum:name_of_callable
where the first part is a filesystem path to the python module and the
where the first part is a filesystem path or url to the python
second part is the name of the callable in the module that will be called.
module and the second part is the name of the callable in the
The callable will be passed three parameters in the following order:
module that will be called. The callable will be passed three
parameters in the following order:
1. The ``options`` dictionary from the recipe.
1. The ``options`` dictionary from the recipe.
...
@@ -121,6 +124,31 @@ Supported options
...
@@ -121,6 +124,31 @@ Supported options
``make``. The format and semantics are the same as with the
``make``. The format and semantics are the same as with the
``pre-configure-hook`` option.
``pre-configure-hook`` option.
.. hook shell command:
``pre-configure``
Shell command that will be executed before running ``configure``
script. It takes the same effect as ``pre-configure-hook`` option
except it's shell command.
``pre-build``
Shell command that will be executed before running ``make``. It
takes the same effect as ``pre-make-hook`` option except it's
shell command.
``pre-install``
Shell command that will be executed before running ``make``
install.
``post-install``
Shell command that will be executed after running ``make``. It
takes the same effect as ``post-make-hook`` option except it's
shell command.
``keep-compile-dir``
``keep-compile-dir``
Switch to optionally keep the temporary directory where the
Switch to optionally keep the temporary directory where the
...
@@ -130,6 +158,16 @@ Supported options
...
@@ -130,6 +158,16 @@ Supported options
compile directory is stored in ``options['compile-directory']``.
compile directory is stored in ``options['compile-directory']``.
Accepted values are ``true`` or ``false``, defaults to ``false``.
Accepted values are ``true`` or ``false``, defaults to ``false``.
``dependencies``
List all the depended parts:
dependencies = part1 part2 ...
All the dependent parts will be installed before this part, besides
the changes in any dependent parts will trigger to reinstall
current part.
``environment-section``
``environment-section``
Name of a section that provides environment variables that will be used to
Name of a section that provides environment variables that will be used to
...
@@ -165,6 +203,29 @@ Supported options
...
@@ -165,6 +203,29 @@ Supported options
both ``environment-section`` and ``environment`` are provided the values from
both ``environment-section`` and ``environment`` are provided the values from
the former will be overridden by the latter allowing per-part customization.
the former will be overridden by the latter allowing per-part customization.
The recipe uses separated part to support custom options in different
platforms. These platform's part has a pattern "platform-part" or
"arch-platform-part".
arch could be 'x86', 'x86_64', 'ia64' ... which equals
platform.machine().
platform could be 'linux', 'cygwin', 'macos', 'sunos', 'freebsd',
'netbsd', 'unixware' ... which equals a formatted sys.platform.
For example,
[bzip2]
recipe = slapos.recipe.cmmi
[x86-cygwin-bzip2]
patches = cygwin-bzip2-1.0.6.src.patch
All the options in the platform-part have high priority level.
The recipe first searches the exact match, if no found. Ignore arch
and search again, if still found nothing. Use no platform part.
Additionally, the recipe honors the ``download-cache`` option set
Additionally, the recipe honors the ``download-cache`` option set
in the ``[buildout]`` section and stores the downloaded files under
in the ``[buildout]`` section and stores the downloaded files under
it. If the value is not set a directory called ``downloads`` will be
it. If the value is not set a directory called ``downloads`` will be
...
@@ -176,6 +237,27 @@ before downloading it from the net. Files can be shared among
...
@@ -176,6 +237,27 @@ before downloading it from the net. Files can be shared among
different buildouts by setting the ``download-cache`` to the same
different buildouts by setting the ``download-cache`` to the same
location.
location.
The recipe honors the ``prefix`` option set in the ``[buildout]``
section either. It implicts all the parts which recipe is
slapos.recipe.cmmi in this buildout process will be installed in the
same ``prefix`` option in the ``[buildout]``. Besides, once it takes
effects, recipe will return all the installed files in the prefix
directory. The own ``prefix`` of part will disable this behaviour.
If the ``buildout`` section has a valid ``prefix`` option, the recipe
will add it to environmet variables as the following:
PATH=${buildout:prefix}/bin:$PATH
CPPFLAGS=-I${buildout:prefix} $CPPFLAGS
CFLAGS=-I${buildout:prefix} $CFFLAGS
CXXFLAGS=-I${buildout:prefix} $CXXFLAGS
LDFLAGS=-L${buildout:prefix}/lib
Besides, the recipe changes environment variable ``TMP`` when building
and installing, and make a corresponding directory 'tmp' in the
``location``. This temporary directory will be removed after
installing finished.
Example usage
Example usage
=============
=============
...
@@ -210,6 +292,7 @@ default build options.
...
@@ -210,6 +292,7 @@ default build options.
>>> print(system(buildout))
>>> print(system(buildout))
Installing package.
Installing package.
package: [ENV] TMP = /sample_buildout/parts/package/tmp
package: Extracting package to /sample_buildout/parts/package__compile__
package: Extracting package to /sample_buildout/parts/package__compile__
configure --prefix=/sample_buildout/parts/package
configure --prefix=/sample_buildout/parts/package
building package
building package
...
@@ -245,6 +328,7 @@ a custom location within the buildout::
...
@@ -245,6 +328,7 @@ a custom location within the buildout::
>>> print(system(buildout))
>>> print(system(buildout))
Uninstalling package.
Uninstalling package.
Installing foobar.
Installing foobar.
foobar: [ENV] TMP = /sample_buildout/parts/foobar/tmp
foobar: Extracting package to /sample_buildout/parts/foobar__compile__
foobar: Extracting package to /sample_buildout/parts/foobar__compile__
building package
building package
installing package
installing package
...
@@ -284,6 +368,7 @@ Makefile and using explicit ``make`` options to control the build process.
...
@@ -284,6 +368,7 @@ Makefile and using explicit ``make`` options to control the build process.
>>> print(system(buildout))
>>> print(system(buildout))
Uninstalling foobar.
Uninstalling foobar.
Installing haproxy.
Installing haproxy.
haproxy: [ENV] TMP = /sample_buildout/parts/haproxy/tmp
haproxy: Extracting package to /sample_buildout/parts/haproxy__compile__
haproxy: Extracting package to /sample_buildout/parts/haproxy__compile__
Building HAProxy 1.4.8 (dummy package)
Building HAProxy 1.4.8 (dummy package)
TARGET: linux26
TARGET: linux26
...
@@ -325,6 +410,7 @@ and building that.
...
@@ -325,6 +410,7 @@ and building that.
>>> print(system(buildout))
>>> print(system(buildout))
Uninstalling haproxy.
Uninstalling haproxy.
Installing package.
Installing package.
package: [ENV] TMP = /sample_buildout/parts/package/tmp
package: Using local source directory: /checkout/package-0.0.0
package: Using local source directory: /checkout/package-0.0.0
configure --prefix=/sample_buildout/parts/package
configure --prefix=/sample_buildout/parts/package
building package
building package
...
@@ -384,6 +470,7 @@ targets and also patches the source code before the scripts are run.
...
@@ -384,6 +470,7 @@ targets and also patches the source code before the scripts are run.
Installing package.
Installing package.
package: [ENV] CFLAGS = -I/sw/include
package: [ENV] CFLAGS = -I/sw/include
package: [ENV] LDFLAGS = -L/sw/lib -L/some/extra/lib
package: [ENV] LDFLAGS = -L/sw/lib -L/some/extra/lib
package: [ENV] TMP = /sample_buildout/parts/package/tmp
package: Extracting package to /sample_buildout/parts/package__compile__
package: Extracting package to /sample_buildout/parts/package__compile__
package: Applying patches
package: Applying patches
patching file configure
patching file configure
...
@@ -467,6 +554,7 @@ and a new buildout to try it out
...
@@ -467,6 +554,7 @@ and a new buildout to try it out
>>> print(system(buildout))
>>> print(system(buildout))
Uninstalling package.
Uninstalling package.
Installing package.
Installing package.
package: [ENV] TMP = /sample_buildout/parts/package/tmp
package: Extracting package to /sample_buildout/parts/package__compile__
package: Extracting package to /sample_buildout/parts/package__compile__
package: Executing pre-configure-hook
package: Executing pre-configure-hook
hook: This is pre-configure-hook!
hook: This is pre-configure-hook!
...
@@ -478,6 +566,464 @@ and a new buildout to try it out
...
@@ -478,6 +566,464 @@ and a new buildout to try it out
package: Executing post-make-hook
package: Executing post-make-hook
hook: This is post-make-hook!
hook: This is post-make-hook!
If you prefer to use shell script, then try these options:
pre-configure
pre-build
pre-install
post-install
Let's create a buildout to use these options.
>>> write('buildout.cfg',
... """
... [buildout]
... newest = false
... parts = package
...
... [package]
... recipe = slapos.recipe.cmmi
... url = file://%s/package-0.0.0.tar.gz
... pre-configure = echo "Configure part: ${:_buildout_section_name_}"
... pre-build = echo "OH OH OH" > a.txt
... pre-install = cat a.txt
... post-install = rm -f a.txt && echo "Finished."
... """ % src)
This will run pre-configure, pre-build, pre-install, post-install as
shell command in the corresponding stage.
>>> print system(buildout)
Uninstalling package.
Installing package.
package: [ENV] TMP = /sample_buildout/parts/package/tmp
package: Extracting package to /sample_buildout/parts/package__compile__
package: Executing pre-configure
Configure part: package
configure --prefix=/sample_buildout/parts/package
package: Executing pre-build
building package
package: Executing pre-install
OH OH OH
installing package
package: Executing post-install
Finished.
Building in multi-platforms
===========================
The recipe can specify build options for each platform. For example,
>>> write('buildout.cfg',
... """
... [buildout]
... newest = false
... parts = package
...
... [package]
... recipe = slapos.recipe.cmmi
... url = file://%s/package-0.0.0.tar.gz
... pre-configure = echo "Configure in common platform"
... post-install = echo "Finished."
...
... [cygwin-package]
... pre-configure = echo "Configure in the CYGWIN platform"
... pre-install = echo "Installing in the CYGWIN"
... post-install = echo -n "CYGWIN " && ${package:post-install}
... """ % src)
In the linux, the recipe gets the options from part 'package', there
are only ``pre-configure`` and ``post-install``. the output will be
#>>> print system(buildout)
Uninstalling package.
Installing package.
package: [ENV] TMP = /sample_buildout/parts/package/tmp
package: Extracting package to /sample_buildout/parts/package__compile__
package: Executing pre-configure
Configure part: Configure in common platform
configure --prefix=/sample_buildout/parts/package
building package
installing package
package: Executing post-install
Finished.
In the cygwin, the recipe merges the options in the parts 'package'
and 'cygwin-package'. The output will be
>>> print system(buildout)
Uninstalling package.
Installing package.
package: [ENV] TMP = /sample_buildout/parts/package/tmp
package: Extracting package to /sample_buildout/parts/package__compile__
package: Executing pre-configure
Configure in the CYGWIN platform
configure --prefix=/sample_buildout/parts/package
building package
package: Executing pre-install
Installing in the CYGWIN
installing package
package: Executing post-install
CYGWIN Finished.
Arch is supported either, For example,
[x86-cygwin-package]
...
The recipe will first search part looks like "arch-platform-part",
then "platform-part".
Union prefix
============
If the recipe finds ``prefix`` option in the section buildout, it will
* First, use this ``prefix`` as configure prefix, if
``configure-command`` isn't set in the part, or ``make-binary``
equals 'make' and ``make-target`` includes pattern '\s+install.*'
* Second, return all the new installed files in the prefix when the
recipe returns after intall.
* Finally, change some environment variables(See first section).
Let's see what happens when set prefix in the buildout section:
>>> write('buildout.cfg',
... """
... [buildout]
... newest = false
... parts = package
... prefix = ${buildout:directory}/mylocal
...
... [package]
... recipe = slapos.recipe.cmmi
... url = file://%s/package-0.0.0.tar.gz
... pre-configure = mkdir -p "${buildout:prefix}"
... """ % src)
>>> print system(buildout)
Uninstalling package.
Installing package.
package: [ENV] CFLAGS = /sample-buildout/mylocal/include
package: [ENV] CPPFLAGS = /sample-buildout/mylocal/include
package: [ENV] CXXFLAGS = /sample-buildout/mylocal/include
package: [ENV] LDFLAGS = /sample-buildout/mylocal/lib
package: [ENV] PATH = /sample_buildout/mylocal/bin:/usr/bin
package: [ENV] TMP = /sample_buildout/parts/package/tmp
package: Extracting package to /sample_buildout/parts/package__compile__
package: Executing pre-configure
configure --prefix=/sample_buildout/mylocal
building package
installing package
Look these environment variables and prefix's value, you know what's
the differences.
If part has its own ``prefix``, it will disable above behavious. For
example,
>>> write('buildout.cfg',
... """
... [buildout]
... newest = false
... parts = package
... prefix = ${buildout:directory}/mylocal
...
... [package]
... recipe = slapos.recipe.cmmi
... prefix = ${buildout:parts-directory}/package
... url = file://%s/package-0.0.0.tar.gz
... pre-configure = rm -rf "${buildout:prefix}"
... post-install = test -d "${buildout:prefix}" || echo "None"
... """ % src)
>>> print system(buildout)
Uninstalling package.
Installing package.
package: [ENV] TMP = /sample_buildout/parts/package/tmp
package: Extracting package to /sample_buildout/parts/package__compile__
package: Executing pre-configure
configure --prefix=/sample_buildout/parts/package
building package
installing package
package: Executing post-install
None
Then no extra environment variables such as CFLAGS etc., and no
${buildout:prefix} directory is created.
The following example shows how to install package, package-2 in one
prefix:
>>> write('buildout.cfg',
... """
... [buildout]
... newest = false
... parts = package package-2
... prefix = ${buildout:directory}/mylocal
...
... [package]
... recipe = slapos.recipe.cmmi
... url = file://%s/package-0.0.0.tar.gz
... pre-install = sleep 2; mkdir -p "${buildout:prefix}" ; echo x >"${buildout:prefix}/a.txt"
... [package-2]
... recipe = slapos.recipe.cmmi
... url = file://%s/package-0.0.0.tar.gz
... pre-install = sleep 2; mkdir -p "${buildout:prefix}" ; echo x >"${buildout:prefix}/b.txt"; echo
... """ % (src, src))
>>> print system(buildout)
Uninstalling package.
Installing package.
package: [ENV] CFLAGS = /sample-buildout/mylocal/include
package: [ENV] CPPFLAGS = /sample-buildout/mylocal/include
package: [ENV] CXXFLAGS = /sample-buildout/mylocal/include
package: [ENV] LDFLAGS = /sample-buildout/mylocal/lib
package: [ENV] PATH = /sample_buildout/mylocal/bin:/usr/bin
package: [ENV] TMP = /sample_buildout/parts/package/tmp
package: Extracting package to /sample_buildout/parts/package__compile__
configure --prefix=/sample_buildout/mylocal
building package
package: Executing pre-install
installing package
Installing package-2.
package-2: [ENV] CFLAGS = /sample-buildout/mylocal/include
package-2: [ENV] CPPFLAGS = /sample-buildout/mylocal/include
package-2: [ENV] CXXFLAGS = /sample-buildout/mylocal/include
package-2: [ENV] LDFLAGS = /sample-buildout/mylocal/lib
package-2: [ENV] PATH = /sample_buildout/mylocal/bin:/usr/bin
package-2: [ENV] TMP = /sample_buildout/parts/package-2/tmp
package-2: Extracting package to /sample_buildout/parts/package-2__compile__
configure --prefix=/sample_buildout/mylocal
building package
package-2: Executing pre-install
installing package
>>> ls('mylocal')
- a.txt
- b.txt
Next we unintall package-2, it should only remove file b.txt:
>>> write('buildout.cfg',
... """
... [buildout]
... newest = false
... parts = package
... prefix = ${buildout:directory}/mylocal
...
... [package]
... recipe = slapos.recipe.cmmi
... url = file://%s/package-0.0.0.tar.gz
... pre-install = sleep 2; mkdir -p "${buildout:prefix}" ; echo x >"${buildout:prefix}/a.txt"
... """ % src)
>>> print system(buildout)
Uninstalling package-2.
Updating package.
>>> ls('mylocal')
- a.txt
Magic prefix
============
If configure-command is set, the recipe wouldn't insert "--prefix"
into configure-options. Then it checks whether both of make-binary and
make-targets aren't set, if so, string "prefix=xxx" will be appended
in the make-targets. xxx is the final prefix of this recipe. We call
it Magic Prefix.
In these options magic prefix can be represented by %(prefix)s:
``onfigure-command`` ``configure-options``
``make-binary`` ``make-options`` ``make-targets``
``pre-configure`` ``pre-build`` ``pre-install`` ``post-install``
For example,
[bzip2]
post-install = rm %(prefix)s/*.h
The other part can refer to magic prefix of this part by
${part:prefix}, it will return the magic prefix, other than literal
value in the part section. For example,
>>> write('buildout.cfg',
... """
... [buildout]
... newest = false
... parts = package package-2
... prefix = /mytemp
...
... [package]
... recipe = slapos.recipe.cmmi
... url = file://%s/package-0.0.0.tar.gz
... configure-command = true
... make-binary = true
...
... [package-2]
... recipe = slapos.recipe.cmmi
... url = file://%s/package-0.0.0.tar.gz
... configure-command = true
... make-binary = true
... post-install = echo package magic prefix is ${package:prefix}
... """ % (src, src))
>>> print system(buildout)
Uninstalling package.
Installing package.
package: [ENV] CFLAGS = -I/mytemp/include
package: [ENV] CPPFLAGS = -I/mytemp/include
package: [ENV] CXXFLAGS = -I/mytemp/include
package: [ENV] LDFLAGS = -L/mytemp/lib
package: [ENV] PATH = /mytemp/bin:/usr/bin
package: [ENV] TMP = /sample_buildout/parts/package/tmp
package: Extracting package to /sample_buildout/parts/package__compile__
Installing package-2.
package-2: [ENV] CFLAGS = -I/mytemp/include
package-2: [ENV] CPPFLAGS = -I/mytemp/include
package-2: [ENV] CXXFLAGS = -I/mytemp/include
package-2: [ENV] LDFLAGS = -L/mytemp/lib
package-2: [ENV] PATH = /mytemp/bin:/usr/bin
package-2: [ENV] TMP = /sample_buildout/parts/package-2/tmp
package-2: Extracting package to /sample_buildout/parts/package-2__compile__
package-2: Executing post-install
package magic prefix is /mytemp
Here it's another sample, we change Makefile before installing so it
can display "prefix" value in the stdout.
>>> write('buildout.cfg',
... """
... [buildout]
... newest = false
... parts = package
...
... [package]
... recipe = slapos.recipe.cmmi
... url = file://%s/package-0.0.0.tar.gz
... configure-command = ./configure
... pre-install = sed -i -e "s/installing package/installing package at \$\$prefix /g" Makefile
... """ % src)
>>> print system(buildout)
Uninstalling package-2.
Uninstalling package.
Installing package.
package: [ENV] TMP = /sample_buildout/parts/package/tmp
package: Extracting package to /sample_buildout/parts/package__compile__
configure
building package
package: Executing pre-install
installing package at /sample_buildout/parts/package
You even can include pattern %(prefix)s in this option, it will be
replaced with the recipe final prefix.
>>> write('buildout.cfg',
... """
... [buildout]
... newest = false
... parts = package
...
... [package]
... recipe = slapos.recipe.cmmi
... url = file://%s/package-0.0.0.tar.gz
... configure-command = ./configure
... make-targets = install-lib prefix=%%(prefix)s
... pre-install = sed -i -e "s/installing package/installing package at \$\$prefix /g" Makefile
... """ % src)
>>> print system(buildout)
Uninstalling package.
Installing package.
package: [ENV] TMP = /sample_buildout/parts/package/tmp
package: Extracting package to /sample_buildout/parts/package__compile__
configure
building package
package: Executing pre-install
installing package at /sample_buildout/parts/package -lib
Extra part dependencies
=======================
The recipe will treat all the parts list in the option
``dependencies`` as dependent parts. zc.buildout will install all the
dependent parts before install this part. For example,
>>> write('buildout.cfg',
... """
... [buildout]
... newest = false
... parts = package
...
... [package]
... recipe = slapos.recipe.cmmi
... dependencies = package-2
... url = file://%s/package-0.0.0.tar.gz
...
... [package-2]
... recipe = slapos.recipe.cmmi
... url = file://%s/package-0.0.0.tar.gz
... """ % (src, src))
Here "package-2" will be installed first, because it's a denpend part
of "package":
>>> print system(buildout)
Uninstalling package.
Installing package-2.
package-2: [ENV] TMP = /sample_buildout/parts/package-2/tmp
package-2: Extracting package to /sample_buildout/parts/package-2__compile__
configure --prefix=/sample_buildout/parts/package-2
building package
installing package
Installing package.
package: [ENV] TMP = /sample_buildout/parts/package/tmp
package: Extracting package to /sample_buildout/parts/package__compile__
configure --prefix=/sample_buildout/parts/package
building package
installing package
Now let's add a new option for "package-2",
>>> write('buildout.cfg',
... """
... [buildout]
... newest = false
... parts = package
...
... [package]
... recipe = slapos.recipe.cmmi
... dependencies = package-2
... url = file://%s/package-0.0.0.tar.gz
...
... [package-2]
... recipe = slapos.recipe.cmmi
... url = file://%s/package-0.0.0.tar.gz
... configure-command = ./configure
... """ % (src, src))
Look, "package" is reinstalled either:
>>> print system(buildout)
Uninstalling package.
Uninstalling package-2.
Installing package-2.
package-2: [ENV] TMP = /sample_buildout/parts/package-2/tmp
package-2: Extracting package to /sample_buildout/parts/package-2__compile__
configure
building package
installing package
Installing package.
package: [ENV] TMP = /sample_buildout/parts/package/tmp
package: Extracting package to /sample_buildout/parts/package__compile__
configure --prefix=/sample_buildout/parts/package
building package
installing package
For even more specific needs you can write your own recipe that uses
For even more specific needs you can write your own recipe that uses
``slapos.recipe.cmmi`` and set the ``keep-compile-dir`` option to ``true``.
``slapos.recipe.cmmi`` and set the ``keep-compile-dir`` option to ``true``.
You can then continue from where this recipe finished by reading the location
You can then continue from where this recipe finished by reading the location
...
...
slapos/recipe/cmmi/__init__.py
View file @
50958d2e
import
errno
import
errno
from
hashlib
import
md5
import
hexagonit.recipe.download
import
hexagonit.recipe.download
import
imp
import
imp
import
logging
import
logging
import
os
import
os
import
pkg_resources
from
platform
import
machine
as
platform_machine
import
re
import
shutil
import
shutil
import
subprocess
import
subprocess
import
sys
import
zc.buildout
import
zc.buildout
from
zc.buildout.download
import
Download
class
Recipe
(
object
):
class
Recipe
(
object
):
...
@@ -16,13 +22,45 @@ class Recipe(object):
...
@@ -16,13 +22,45 @@ class Recipe(object):
self
.
buildout
=
buildout
self
.
buildout
=
buildout
self
.
name
=
name
self
.
name
=
name
# Merge options if there is a matched platform section
platform_options
=
self
.
get_platform_options
()
if
platform_options
is
None
:
self
.
original_options
=
options
else
:
self
.
original_options
=
options
.
copy
()
options
.
update
(
platform_options
)
options
[
'location'
]
=
os
.
path
.
join
(
options
[
'location'
]
=
os
.
path
.
join
(
buildout
[
'buildout'
][
'parts-directory'
],
buildout
[
'buildout'
][
'parts-directory'
],
self
.
name
)
self
.
name
)
options
[
'prefix'
]
=
options
.
get
(
'prefix'
,
options
[
'location'
])
prefix
=
options
.
get
(
'prefix'
,
''
).
strip
()
if
prefix
==
''
:
prefix
=
self
.
buildout_prefix
=
buildout
[
'buildout'
].
get
(
'prefix'
,
''
).
strip
()
else
:
self
.
buildout_prefix
=
''
options
[
'prefix'
]
=
options
[
'location'
]
if
prefix
==
''
else
prefix
options
[
'url'
]
=
options
.
get
(
'url'
,
''
).
strip
()
options
[
'url'
]
=
options
.
get
(
'url'
,
''
).
strip
()
options
[
'path'
]
=
options
.
get
(
'path'
,
''
).
strip
()
options
[
'path'
]
=
options
.
get
(
'path'
,
''
).
strip
()
# Check dependencies, all the dependent parts will be installed first. It
# seems once part is referenced, for example, self.buildout[part], it will
# be appended as an install part.
# dpendent_parts = options.get('dependencies', '').split()
# assert isinstance(buildout, zc.buildout.buildout.Buildout)
# buildout._compute_part_signatures(
# [part for part in dpendent_parts if part not in buildout._parts])
# Calculate md5sum of all the options for each dependent part, and save it
# as option "dependencies". So if any dependent part changes, zc.buildout
# will reinstall the part because this option is changed.
dependencies
=
[]
for
part
in
options
.
get
(
'dependencies'
,
''
).
split
():
m
=
md5
()
for
(
k
,
v
)
in
self
.
buildout
[
part
].
iteritems
():
m
.
update
(
v
)
dependencies
.
append
(
m
.
hexdigest
())
options
[
'dependencies'
]
=
' '
.
join
(
dependencies
)
if
options
[
'url'
]
and
options
[
'path'
]:
if
options
[
'url'
]
and
options
[
'path'
]:
raise
zc
.
buildout
.
UserError
(
'You must use either "url" or "path", not both!'
)
raise
zc
.
buildout
.
UserError
(
'You must use either "url" or "path", not both!'
)
if
not
(
options
[
'url'
]
or
options
[
'path'
]):
if
not
(
options
[
'url'
]
or
options
[
'path'
]):
...
@@ -47,10 +85,19 @@ class Recipe(object):
...
@@ -47,10 +85,19 @@ class Recipe(object):
self
.
environ
[
key
.
strip
()]
=
value
self
.
environ
[
key
.
strip
()]
=
value
except
ValueError
:
except
ValueError
:
raise
zc
.
buildout
.
UserError
(
'Invalid environment variable definition: %s'
,
variable
)
raise
zc
.
buildout
.
UserError
(
'Invalid environment variable definition: %s'
,
variable
)
# Add prefix to PATH, CPPFLAGS, CFLAGS, CXXFLAGS, LDFLAGS
if
self
.
buildout_prefix
!=
''
:
self
.
environ
[
'PATH'
]
=
'%s/bin:%s'
%
(
self
.
buildout_prefix
,
self
.
environ
.
get
(
'PATH'
,
'/usr/bin'
))
self
.
environ
[
'CPPFLAGS'
]
=
'-I%s/include %s'
%
(
self
.
buildout_prefix
,
self
.
environ
.
get
(
'CPPFLAGS'
,
''
))
self
.
environ
[
'CFLAGS'
]
=
'-I%s/include %s'
%
(
self
.
buildout_prefix
,
self
.
environ
.
get
(
'CFLAGS'
,
''
))
self
.
environ
[
'CXXFLAGS'
]
=
'-I%s/include %s'
%
(
self
.
buildout_prefix
,
self
.
environ
.
get
(
'CXXFLAGS'
,
''
))
self
.
environ
[
'LDFLAGS'
]
=
'-L%s/lib %s'
%
(
self
.
buildout_prefix
,
self
.
environ
.
get
(
'LDFLAGS'
,
''
))
# Extrapolate the environment variables using values from the current
# Extrapolate the environment variables using values from the current
# environment.
# environment.
for
key
in
self
.
environ
:
for
key
in
self
.
environ
:
self
.
environ
[
key
]
=
self
.
environ
[
key
]
%
os
.
environ
self
.
environ
[
key
]
=
self
.
environ
[
key
]
%
os
.
environ
self
.
environ
[
'TMP'
]
=
os
.
path
.
join
(
options
[
'location'
],
'tmp'
)
def
augmented_environment
(
self
):
def
augmented_environment
(
self
):
"""Returns a dictionary containing the current environment variables
"""Returns a dictionary containing the current environment variables
...
@@ -66,12 +113,79 @@ class Recipe(object):
...
@@ -66,12 +113,79 @@ class Recipe(object):
def
update
(
self
):
def
update
(
self
):
pass
pass
def
_compute_part_signatures
(
self
,
options
):
# Copy from zc.buildout.Buildout, compute recipe signature
recipe
,
entry
=
zc
.
buildout
.
buildout
.
_recipe
(
options
)
req
=
pkg_resources
.
Requirement
.
parse
(
recipe
)
sig
=
zc
.
buildout
.
buildout
.
_dists_sig
(
pkg_resources
.
working_set
.
resolve
([
req
]))
return
' '
.
join
(
sig
)
def
get_platform
(
self
):
# Get value of sys.platform
for
platform
in
[
'linux'
,
'cygwin'
,
'beos'
,
'darwin'
,
'atheos'
,
'osf1'
,
'netbsd'
,
'openbsd'
,
'freebsd'
,
'unixware'
,
'sunos'
]:
if
sys
.
platform
.
startswith
(
platform
):
return
platform
return
sys
.
platform
def
get_machine
(
self
):
arch
=
platform_machine
()
# i?86-*-* : x86
if
arch
in
(
'i386'
,
'i586'
,
'i686'
):
return
'x86'
# x86_64-*-* : amd64
elif
arch
==
'x86_64'
:
return
'amd64'
# ia64-*-* : ia64
# and others
return
arch
def
get_platform_options
(
self
):
platform_part
=
self
.
get_platform
()
+
'-'
+
self
.
name
part_list
=
[
part
for
part
in
self
.
buildout
if
part
.
endswith
(
platform_part
)]
if
part_list
[:
1
]:
arch_prefix
=
self
.
get_machine
()
+
'-'
for
part
in
part_list
:
if
part
.
startswith
(
arch_prefix
):
return
self
.
buildout
[
part
]
else
:
return
self
.
buildout
.
get
(
platform_part
)
def
download_file
(
self
,
url
):
download
=
Download
(
self
.
buildout
[
'buildout'
])
url
,
_s_
,
md5sum
=
url
.
partition
(
'#'
)
return
download
(
url
,
md5sum
=
None
if
md5sum
==
''
else
md5sum
)
def
get_installed_files
(
self
,
ref_file
):
# if [buildout] has option 'prefix', then return all the files
# in this path which create time is newer than ref_file.
# Exclude directory and don't follow link.
assert
self
.
buildout_prefix
log
=
logging
.
getLogger
(
self
.
name
)
cmd
=
'find %s -cnewer %s ! -type d'
%
(
self
.
buildout_prefix
,
ref_file
)
try
:
p
=
subprocess
.
Popen
(
cmd
,
shell
=
True
,
stdout
=
subprocess
.
PIPE
)
files
,
_i_
=
p
.
communicate
()
retcode
=
p
.
returncode
if
retcode
<
0
:
log
.
error
(
'Command received signal %s: %s'
%
(
-
retcode
,
cmd
))
raise
zc
.
buildout
.
UserError
(
'System error'
)
elif
retcode
>
0
:
log
.
error
(
'Command failed with exit code %s: %s'
%
(
retcode
,
cmd
))
raise
zc
.
buildout
.
UserError
(
'System error'
)
except
OSError
,
e
:
log
.
error
(
'Command failed: %s: %s'
%
(
e
,
cmd
))
raise
zc
.
buildout
.
UserError
(
'System error'
)
return
files
.
split
()
def
call_script
(
self
,
script
):
def
call_script
(
self
,
script
):
"""This method is copied from z3c.recipe.runscript.
"""This method is copied from z3c.recipe.runscript.
See http://pypi.python.org/pypi/z3c.recipe.runscript for details.
See http://pypi.python.org/pypi/z3c.recipe.runscript for details.
"""
"""
filename
,
callable
=
script
.
rsplit
(
':'
,
1
)
url
,
callable
=
script
.
rsplit
(
':'
,
1
)
filename
,
is_temp
=
self
.
download_file
(
url
)
if
not
is_temp
:
filename
=
os
.
path
.
abspath
(
filename
)
filename
=
os
.
path
.
abspath
(
filename
)
module
=
imp
.
load_source
(
'script'
,
filename
)
module
=
imp
.
load_source
(
'script'
,
filename
)
script
=
getattr
(
module
,
callable
.
strip
())
script
=
getattr
(
module
,
callable
.
strip
())
...
@@ -82,6 +196,9 @@ class Recipe(object):
...
@@ -82,6 +196,9 @@ class Recipe(object):
# BBB: Support hook scripts that do not take the environment as
# BBB: Support hook scripts that do not take the environment as
# the third parameter
# the third parameter
script
(
self
.
options
,
self
.
buildout
)
script
(
self
.
options
,
self
.
buildout
)
finally
:
if
is_temp
:
os
.
remove
(
filename
)
def
run
(
self
,
cmd
):
def
run
(
self
,
cmd
):
"""Run the given ``cmd`` in a child process."""
"""Run the given ``cmd`` in a child process."""
...
@@ -116,6 +233,8 @@ class Recipe(object):
...
@@ -116,6 +233,8 @@ class Recipe(object):
# Inject the --prefix parameter if not already present
# Inject the --prefix parameter if not already present
if
'--prefix'
not
in
' '
.
join
(
configure_options
):
if
'--prefix'
not
in
' '
.
join
(
configure_options
):
configure_options
.
insert
(
0
,
'--prefix=
\
"
%s
\
"
'
%
self
.
options
[
'prefix'
])
configure_options
.
insert
(
0
,
'--prefix=
\
"
%s
\
"
'
%
self
.
options
[
'prefix'
])
elif
make_cmd
==
'make'
and
make_targets
==
'install'
:
make_targets
+=
' prefix=
\
"
%s
\
"
'
%
self
.
options
[
'prefix'
]
patch_cmd
=
self
.
options
.
get
(
'patch-binary'
,
'patch'
).
strip
()
patch_cmd
=
self
.
options
.
get
(
'patch-binary'
,
'patch'
).
strip
()
patch_options
=
' '
.
join
(
self
.
options
.
get
(
'patch-options'
,
'-p0'
).
split
())
patch_options
=
' '
.
join
(
self
.
options
.
get
(
'patch-options'
,
'-p0'
).
split
())
...
@@ -152,6 +271,9 @@ class Recipe(object):
...
@@ -152,6 +271,9 @@ class Recipe(object):
if
e
.
errno
==
errno
.
EEXIST
:
if
e
.
errno
==
errno
.
EEXIST
:
pass
pass
os
.
chdir
(
compile_dir
)
os
.
chdir
(
compile_dir
)
tmp_path
=
self
.
environ
[
'TMP'
]
shutil
.
rmtree
(
tmp_path
,
True
)
os
.
mkdir
(
tmp_path
)
try
:
try
:
try
:
try
:
...
@@ -165,34 +287,62 @@ class Recipe(object):
...
@@ -165,34 +287,62 @@ class Recipe(object):
if
patches
:
if
patches
:
log
.
info
(
'Applying patches'
)
log
.
info
(
'Applying patches'
)
for
patch
in
patches
:
for
patch
in
patches
:
self
.
run
(
'%s %s < %s'
%
(
patch_cmd
,
patch_options
,
patch
))
patch_filename
,
is_temp
=
self
.
download_file
(
patch
)
self
.
run
(
'%s %s < %s'
%
(
patch_cmd
,
patch_options
,
patch_filename
))
if
is_temp
:
os
.
remove
(
path_filename
)
if
'pre-configure-hook'
in
self
.
options
and
len
(
self
.
options
[
'pre-configure-hook'
].
strip
())
>
0
:
if
'pre-configure-hook'
in
self
.
options
and
len
(
self
.
options
[
'pre-configure-hook'
].
strip
())
>
0
:
log
.
info
(
'Executing pre-configure-hook'
)
log
.
info
(
'Executing pre-configure-hook'
)
self
.
call_script
(
self
.
options
[
'pre-configure-hook'
])
self
.
call_script
(
self
.
options
[
'pre-configure-hook'
])
self
.
run
(
'%s %s'
%
(
configure_cmd
,
' '
.
join
(
configure_options
)))
pre_configure_cmd
=
self
.
options
.
get
(
'pre-configure'
,
''
).
strip
()
%
self
.
options
if
pre_configure_cmd
!=
''
:
log
.
info
(
'Executing pre-configure'
)
self
.
run
(
pre_configure_cmd
)
self
.
run
((
'%s %s'
%
(
configure_cmd
,
' '
.
join
(
configure_options
)))
%
self
.
options
)
if
'pre-make-hook'
in
self
.
options
and
len
(
self
.
options
[
'pre-make-hook'
].
strip
())
>
0
:
if
'pre-make-hook'
in
self
.
options
and
len
(
self
.
options
[
'pre-make-hook'
].
strip
())
>
0
:
log
.
info
(
'Executing pre-make-hook'
)
log
.
info
(
'Executing pre-make-hook'
)
self
.
call_script
(
self
.
options
[
'pre-make-hook'
])
self
.
call_script
(
self
.
options
[
'pre-make-hook'
])
self
.
run
(
'%s %s'
%
(
make_cmd
,
make_options
))
pre_build_cmd
=
self
.
options
.
get
(
'pre-build'
,
''
).
strip
()
%
self
.
options
self
.
run
(
'%s %s %s'
%
(
make_cmd
,
make_options
,
make_targets
))
if
pre_build_cmd
!=
''
:
log
.
info
(
'Executing pre-build'
)
self
.
run
(
pre_build_cmd
)
self
.
run
((
'%s %s'
%
(
make_cmd
,
make_options
))
%
self
.
options
)
pre_install_cmd
=
self
.
options
.
get
(
'pre-install'
,
''
).
strip
()
%
self
.
options
if
pre_install_cmd
!=
''
:
log
.
info
(
'Executing pre-install'
)
self
.
run
(
pre_install_cmd
)
self
.
run
((
'%s %s %s'
%
(
make_cmd
,
make_options
,
make_targets
))
%
self
.
options
)
if
'post-make-hook'
in
self
.
options
and
len
(
self
.
options
[
'post-make-hook'
].
strip
())
>
0
:
if
'post-make-hook'
in
self
.
options
and
len
(
self
.
options
[
'post-make-hook'
].
strip
())
>
0
:
log
.
info
(
'Executing post-make-hook'
)
log
.
info
(
'Executing post-make-hook'
)
self
.
call_script
(
self
.
options
[
'post-make-hook'
])
self
.
call_script
(
self
.
options
[
'post-make-hook'
])
post_install_cmd
=
self
.
options
.
get
(
'post-install'
,
''
).
strip
()
%
self
.
options
if
post_install_cmd
!=
''
:
log
.
info
(
'Executing post-install'
)
self
.
run
(
post_install_cmd
)
if
self
.
buildout_prefix
!=
''
and
os
.
path
.
exists
(
self
.
buildout_prefix
):
log
.
info
(
'Getting installed file lists'
)
parts
.
extend
(
self
.
get_installed_files
(
tmp_path
))
except
:
except
:
log
.
error
(
'Compilation error. The package is left as is at %s where '
log
.
error
(
'Compilation error. The package is left as is at %s where '
'you can inspect what went wrong'
%
os
.
getcwd
())
'you can inspect what went wrong'
%
os
.
getcwd
())
raise
raise
finally
:
finally
:
os
.
chdir
(
current_dir
)
os
.
chdir
(
current_dir
)
shutil
.
rmtree
(
tmp_path
)
if
self
.
options
[
'url'
]:
if
self
.
options
[
'url'
]:
if
self
.
options
.
get
(
'keep-compile-dir'
,
''
).
lower
()
in
(
'true'
,
'yes'
,
'1'
,
'on'
):
if
self
.
options
.
get
(
'keep-compile-dir'
,
self
.
buildout
[
'buildout'
].
get
(
'keep-compile-dir'
,
''
)).
lower
()
in
(
'true'
,
'yes'
,
'1'
,
'on'
):
# If we're keeping the compile directory around, add it to
# If we're keeping the compile directory around, add it to
# the parts so that it's also removed when this recipe is
# the parts so that it's also removed when this recipe is
# uninstalled.
# uninstalled.
...
...
slapos/recipe/cmmi/tests.py
View file @
50958d2e
...
@@ -60,6 +60,7 @@ class NonInformativeTests(unittest.TestCase):
...
@@ -60,6 +60,7 @@ class NonInformativeTests(unittest.TestCase):
}
}
}
}
bo
.
update
(
buildout
)
bo
.
update
(
buildout
)
bo
[
'buildout'
].
update
(
buildout_options
)
return
Recipe
(
bo
,
name
,
options
)
return
Recipe
(
bo
,
name
,
options
)
def
test_working_directory_restored_after_failure
(
self
):
def
test_working_directory_restored_after_failure
(
self
):
...
@@ -199,6 +200,111 @@ class NonInformativeTests(unittest.TestCase):
...
@@ -199,6 +200,111 @@ class NonInformativeTests(unittest.TestCase):
except
ValueError
as
e
:
except
ValueError
as
e
:
self
.
assertEqual
(
str
(
e
),
'sentinel bar'
)
self
.
assertEqual
(
str
(
e
),
'sentinel bar'
)
def
no_test_make_target_with_prefix
(
self
):
recipe
=
self
.
make_recipe
({},
'test'
,
{
'url'
:
'file://%s/testdata/package-0.0.0.tar.gz'
%
os
.
path
.
dirname
(
__file__
),
'configure-command'
:
'./configure'
,
'pre-install'
:
'sed -i -e "s/installing package/installing package at
\
$
\
$prefix /g" Makefile'
,
})
os
.
chdir
(
self
.
dir
)
recipe
.
install
()
recipe
=
self
.
make_recipe
({},
'test'
,
{
'url'
:
'file://%s/testdata/package-0.0.0.tar.gz'
%
os
.
path
.
dirname
(
__file__
),
'pre-install'
:
'sed -i -e "s/installing package/installing package at
\
$
\
$prefix /g" Makefile'
,
'make-targets'
:
'install-lib prefix=%(prefix)s'
,
})
recipe
.
install
()
def
test_download_file
(
self
):
recipe
=
self
.
make_recipe
({},
'test'
,
{
'url'
:
'file://%s/testdata/package-0.0.0.tar.gz'
%
os
.
path
.
dirname
(
__file__
),
})
url
=
'%s/testdata/package-0.0.0.tar.gz'
%
os
.
path
.
dirname
(
__file__
)
file
,
is_temp
=
recipe
.
download_file
(
url
)
self
.
assertFalse
(
is_temp
)
self
.
assertEquals
(
file
,
url
)
url
=
'ftp://ftp.gnu.org/welcome.msg'
file
,
is_temp
=
recipe
.
download_file
(
url
)
self
.
assertTrue
(
is_temp
)
self
.
assertTrue
(
os
.
path
.
exists
(
file
))
url
=
'ftp://ftp.gnu.org/welcome.msg#ec7ab8024467ba3b6e173c57fd4990f6'
file
,
is_temp
=
recipe
.
download_file
(
url
)
self
.
assertTrue
(
is_temp
)
self
.
assertTrue
(
os
.
path
.
exists
(
file
))
url
=
'ftp://ftp.gnu.org/welcome.msg#0205'
self
.
assertRaises
(
zc
.
buildout
.
download
.
ChecksumError
,
recipe
.
download_file
,
url
)
def
test_buildout_prefix
(
self
):
buildout_prefix
=
os
.
path
.
join
(
self
.
dir
,
'test_parts/test_local'
)
os
.
makedirs
(
buildout_prefix
)
recipe
=
self
.
make_recipe
({},
'test'
,
{
'url'
:
'file://%s/testdata/package-0.0.0.tar.gz'
%
os
.
path
.
dirname
(
__file__
),
},
prefix
=
buildout_prefix
)
self
.
assertEquals
(
recipe
.
options
.
get
(
'prefix'
),
buildout_prefix
)
recipe
=
self
.
make_recipe
({},
'test'
,
{
'url'
:
'file://%s/testdata/package-0.0.0.tar.gz'
%
os
.
path
.
dirname
(
__file__
),
'prefix'
:
self
.
dir
,
},
prefix
=
buildout_prefix
)
self
.
assertEquals
(
recipe
.
options
.
get
(
'prefix'
),
self
.
dir
)
def
test_get_installed_files
(
self
):
prefix
=
os
.
path
.
join
(
self
.
dir
,
'test_parts/test_local'
)
os
.
makedirs
(
prefix
)
recipe
=
self
.
make_recipe
({},
'test'
,
{
'url'
:
'file://%s/testdata/package-0.0.0.tar.gz'
%
os
.
path
.
dirname
(
__file__
),
})
os
.
chdir
(
self
.
dir
)
# The hook script does not return anything so we (ab)use exceptions
# as a mechanism for asserting the function behaviour.
no_installed_files
=
(
'a.txt'
,
'b.txt'
,
'c'
,
'c/d.txt'
)
installed_files
=
[
'e.txt'
,
'f.txt'
,
'g'
,
'g/h.txt'
]
for
s
in
no_installed_files
:
if
s
.
endswith
(
'.txt'
):
f
=
open
(
os
.
path
.
join
(
prefix
,
s
),
'w'
)
f
.
write
(
s
)
f
.
close
()
else
:
os
.
makedirs
(
os
.
path
.
join
(
prefix
,
s
))
sleep
(
2
)
ref_path
=
os
.
path
.
join
(
self
.
dir
,
'refs'
)
os
.
makedirs
(
ref_path
)
sleep
(
2
)
for
s
in
installed_files
:
if
s
.
endswith
(
'.txt'
):
f
=
open
(
os
.
path
.
join
(
prefix
,
s
),
'w'
)
f
.
write
(
s
)
f
.
close
()
else
:
os
.
makedirs
(
os
.
path
.
join
(
prefix
,
s
))
recipe
.
buildout_prefix
=
prefix
file_list
=
recipe
.
get_installed_files
(
ref_path
)
installed_files
.
pop
(
2
)
self
.
assertEquals
([
os
.
path
.
relpath
(
f
,
prefix
)
for
f
in
file_list
],
installed_files
)
def
test_honor_buidlout_keep_compile_directory
(
self
):
buildout
=
{
'keep-compile-dir'
:
'true'
}
recipe
=
self
.
make_recipe
({},
'test'
,
{
'url'
:
'file://%s/testdata/package-0.0.0.tar.gz'
%
os
.
path
.
dirname
(
__file__
),
},
**
buildout
)
os
.
chdir
(
self
.
dir
)
recipe
.
install
()
build_directory
=
os
.
path
.
join
(
self
.
dir
,
'test_parts/test__compile__'
)
self
.
assertTrue
(
os
.
path
.
exists
(
build_directory
))
def
test_suite
():
def
test_suite
():
suite
=
unittest
.
TestSuite
((
suite
=
unittest
.
TestSuite
((
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment