Skip to content

slapos
slapos
Commit 4f1be677 authored by Jérome Perrin's avatar Jérome Perrin
Browse files
Options

Simplify installation of recipe installing scripts 

Several software use `slapos` command and expect this command to have networkcache support enabled, which is achieved in buildout by having `slapos.libnetworkcache` in the egg "working set" when generating the slapos script.

Introduce a dedicated section to install `slapos` command and use it in softwares which includes a slapos, testnode and slaprunner. Test softwares like slapos-sr-testing also need a slapos command, but they generate one with development version of slapos.core.

Before these changes, several software were installing `slapos` indirectly by installing `extra-eggs` from monitoring stack. The monitoring stack was also modified so that it uses a section with a prefix (`monitor-eggs`) and users of monitoring stacks were also modified to not use this `extra-eggs` part from monitoring stack, but install eggs through their own section.

Similar problem also existed with `slapos.toolbox`, which is installed by several softwares and stacks, for different reasons (because it contains lots of utilities), now that buildout detects parts installing the same path, only one parts can install the scripts. For this, define a section `slapos-toolbox` that software and stack should install, instead of listing the egg as `eggs` in a `zc.recipe.egg` section, because that would install the scripts again.

See merge request !729
parents 6b426c23 3fd2d51e
Hide whitespace changes
Inline Side-by-side
Showing with 66 additions and 68 deletions
software/apache-frontend/common.cfg
View file @ 4f1be677
...@@ -38,9 +38,9 @@ parts += ...@@ -38,9 +38,9 @@ parts +=
proxy-by-url proxy-by-url
http-proxy http-proxy
# Extent extra-eggs.
[extra-eggs] [extra-eggs]
eggs += recipe = zc.recipe.egg
eggs =
websockify websockify
erp5.util erp5.util
......
software/caddy-frontend/common.cfg
View file @ 4f1be677
...@@ -73,12 +73,7 @@ depends = ${caddyprofiledeps-develop:recipe} ...@@ -73,12 +73,7 @@ depends = ${caddyprofiledeps-develop:recipe}
recipe = zc.recipe.egg recipe = zc.recipe.egg
eggs = eggs =
caddyprofiledeps caddyprofiledeps
# Extent extra-eggs.
[extra-eggs]
eggs +=
websockify websockify
erp5.util
collective.recipe.shelloutput collective.recipe.shelloutput
[template-common] [template-common]
......
software/erp5testnode/software.cfg
View file @ 4f1be677
...@@ -14,9 +14,9 @@ extends = ...@@ -14,9 +14,9 @@ extends =
../../stack/monitor/buildout.cfg ../../stack/monitor/buildout.cfg
parts = parts =
slapos-command
slapos-cookbook slapos-cookbook
template template
lxml-python
eggs eggs
zip zip
git git
...@@ -25,25 +25,8 @@ parts = ...@@ -25,25 +25,8 @@ parts =
[eggs] [eggs]
recipe = zc.recipe.egg recipe = zc.recipe.egg
eggs = eggs =
erp5.util[testnode]
${lxml-python:egg} ${lxml-python:egg}
zc.buildout
slapos.recipe.template
supervisor
jsonschema
hexagonit.recipe.download
lock_file
pytz
erp5.util
PyXML
z3c.etestbrowser
[testnode]
scripts =
testnode = erp5.util.testnode:main
slapgrid-cp = slapos.grid.slapgrid:runComputerPartition
slapgrid-sr = slapos.grid.slapgrid:runSoftwareRelease
slapproxy = slapos.proxy:main
[template] [template]
......
software/gateone/software.cfg
View file @ 4f1be677
...@@ -15,7 +15,7 @@ extends = ...@@ -15,7 +15,7 @@ extends =
../../component/nginx/buildout.cfg ../../component/nginx/buildout.cfg
../../stack/slapos.cfg ../../stack/slapos.cfg
parts = parts =
slapos-cookbook slapos-cookbook
kerberos kerberos
eggs eggs
...@@ -24,7 +24,7 @@ parts = ...@@ -24,7 +24,7 @@ parts =
dash dash
check-recipe check-recipe
template template
[eggs] [eggs]
recipe = zc.recipe.egg recipe = zc.recipe.egg
...@@ -91,7 +91,6 @@ extra-context = ...@@ -91,7 +91,6 @@ extra-context =
key template_logrotate_base template-logrotate-base:rendered key template_logrotate_base template-logrotate-base:rendered
key template_gateone template-gateone:target key template_gateone template-gateone:target
raw gateone_bin ${buildout:bin-directory}/gateone raw gateone_bin ${buildout:bin-directory}/gateone
raw python_with_eggs ${buildout:directory}/bin/${extra-eggs:interpreter}
[template-gateone] [template-gateone]
< = download-base < = download-base
......
software/kvm/buildout.hash.cfg
View file @ 4f1be677
...@@ -15,7 +15,7 @@ ...@@ -15,7 +15,7 @@
[template] [template]
filename = instance.cfg.in filename = instance.cfg.in
md5sum = cad7d490b84a484b089e34b27a5675ff md5sum = 763f66b915839a9b073fb93f43a44493
[template-kvm] [template-kvm]
filename = instance-kvm.cfg.jinja2 filename = instance-kvm.cfg.jinja2
......
software/kvm/instance.cfg.in
View file @ 4f1be677
...@@ -26,10 +26,10 @@ RootSoftwareInstance = $${:default} ...@@ -26,10 +26,10 @@ RootSoftwareInstance = $${:default}
# XXX - If this configuration is not generated by slapgrid, use empty values # XXX - If this configuration is not generated by slapgrid, use empty values
[storage-configuration] [storage-configuration]
storage-home = storage-home =
[network-information] [network-information]
global-ipv4-network = global-ipv4-network =
[slap-configuration] [slap-configuration]
recipe = slapos.cookbook:slapconfiguration.serialised recipe = slapos.cookbook:slapconfiguration.serialised
...@@ -87,7 +87,7 @@ extra-context = ...@@ -87,7 +87,7 @@ extra-context =
raw novnc_location ${noVNC:location} raw novnc_location ${noVNC:location}
raw netcat_bin ${netcat:location}/bin/netcat raw netcat_bin ${netcat:location}/bin/netcat
raw python_executable ${buildout:executable} raw python_executable ${buildout:executable}
raw python_eggs_executable ${buildout:bin-directory}/${extra-eggs:interpreter} raw python_eggs_executable ${buildout:bin-directory}/${python-with-eggs:interpreter}
raw qemu_executable_location ${kvm:location}/bin/qemu-system-x86_64 raw qemu_executable_location ${kvm:location}/bin/qemu-system-x86_64
raw qemu_img_executable_location ${kvm:location}/bin/qemu-img raw qemu_img_executable_location ${kvm:location}/bin/qemu-img
raw qemu_start_promise_tpl ${template-qemu-ready:location}/${template-qemu-ready:filename} raw qemu_start_promise_tpl ${template-qemu-ready:location}/${template-qemu-ready:filename}
......
software/kvm/software.cfg
View file @ 4f1be677
...@@ -8,6 +8,7 @@ extends = ...@@ -8,6 +8,7 @@ extends =
../../component/noVNC/buildout.cfg ../../component/noVNC/buildout.cfg
../../component/openssl/buildout.cfg ../../component/openssl/buildout.cfg
../../component/netcat/buildout.cfg ../../component/netcat/buildout.cfg
../../component/lxml-python/buildout.cfg
../../component/pycurl/buildout.cfg ../../component/pycurl/buildout.cfg
../../component/gzip/buildout.cfg ../../component/gzip/buildout.cfg
../../stack/slapos.cfg ../../stack/slapos.cfg
...@@ -40,9 +41,12 @@ parts = ${:common-parts} ...@@ -40,9 +41,12 @@ parts = ${:common-parts}
#XXX-Cedric : add list of keyboard layouts (azerty/us querty/...) parameter to qemu #XXX-Cedric : add list of keyboard layouts (azerty/us querty/...) parameter to qemu
[extra-eggs] [python-with-eggs]
eggs += recipe = zc.recipe.egg
interpreter = ${:_buildout_section_name_}
eggs =
${python-cffi:egg} ${python-cffi:egg}
${lxml-python:egg}
websockify websockify
slapos.cookbook slapos.cookbook
erp5.util erp5.util
......
software/nginx-push-stream/software.cfg
View file @ 4f1be677
[buildout] [buildout]
extends = extends =
../../stack/slapos.cfg ../../stack/slapos.cfg
../../component/dash/buildout.cfg ../../component/dash/buildout.cfg
../../component/nginx/buildout.cfg ../../component/nginx/buildout.cfg
parts = parts =
slapos-cookbook slapos-cookbook
slapos-toolbox
nginx-push-stream-module nginx-push-stream-module
nginx-push-stream nginx-push-stream
template template
template-nginx-service template-nginx-service
template-nginx template-nginx
[slapos-cookbook]
eggs +=
slapos.toolbox
[template] [template]
recipe = slapos.recipe.template recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance.cfg.in url = ${:_profile_base_location_}/instance.cfg.in
......
software/slaprunner/buildout.hash.cfg
View file @ 4f1be677
...@@ -18,7 +18,7 @@ md5sum = ec70348dd71b319590a5c5837f3d2e45 ...@@ -18,7 +18,7 @@ md5sum = ec70348dd71b319590a5c5837f3d2e45
[template-runner] [template-runner]
filename = instance-runner.cfg filename = instance-runner.cfg
md5sum = dc3a7734b08977f438b35d441a12f628 md5sum = f342daca2e87f8fa44fd7b6a2a1c22f8
[template-runner-import-script] [template-runner-import-script]
filename = template/runner-import.sh.jinja2 filename = template/runner-import.sh.jinja2
......
software/slaprunner/instance-runner.cfg
View file @ 4f1be677
...@@ -811,7 +811,7 @@ mode = 0744 ...@@ -811,7 +811,7 @@ mode = 0744
context = context =
section supervisord supervisord section supervisord supervisord
section slaprunner slaprunner section slaprunner slaprunner
raw python_executable ${buildout:bin-directory}/${extra-eggs:interpreter} raw python_executable ${buildout:bin-directory}/${python-with-eggs:interpreter}
[supervisord-wrapper] [supervisord-wrapper]
recipe = slapos.cookbook:wrapper recipe = slapos.cookbook:wrapper
......
software/slaprunner/software-py3.cfg
View file @ 4f1be677
...@@ -9,7 +9,7 @@ python = python3 ...@@ -9,7 +9,7 @@ python = python3
common-parts -= common-parts -=
rdiff-backup rdiff-backup
[extra-eggs] [python-with-eggs]
eggs -= eggs -=
# futures is a backport of Py3's concurrent.futures module # futures is a backport of Py3's concurrent.futures module
futures futures
software/slaprunner/software.cfg
View file @ 4f1be677
...@@ -37,6 +37,7 @@ shared-part-list = ...@@ -37,6 +37,7 @@ shared-part-list =
# to avoid versioning issues # to avoid versioning issues
common-parts = common-parts =
slapos-command
slapos-cookbook slapos-cookbook
template template
instance-runner-import instance-runner-import
...@@ -50,6 +51,7 @@ common-parts = ...@@ -50,6 +51,7 @@ common-parts =
parts = parts =
${:common-parts} ${:common-parts}
[template-base] [template-base]
recipe = slapos.recipe.template recipe = slapos.recipe.template
url = ${:_profile_base_location_}/${:filename} url = ${:_profile_base_location_}/${:filename}
...@@ -141,8 +143,11 @@ filename = slapos-slapuser-script.in ...@@ -141,8 +143,11 @@ filename = slapos-slapuser-script.in
< = template-base < = template-base
output = ${buildout:directory}/buildout-shared-part-list output = ${buildout:directory}/buildout-shared-part-list
[extra-eggs] [python-with-eggs]
eggs += recipe = zc.recipe.egg
interpreter = ${:_buildout_section_name_}
eggs =
${slapos-toolbox:eggs}
erp5.util erp5.util
lock-file lock-file
slapos.recipe.build slapos.recipe.build
...@@ -154,6 +159,11 @@ eggs += ...@@ -154,6 +159,11 @@ eggs +=
# BBB: eggs used as recipe should be kept otherwise sections depending # BBB: eggs used as recipe should be kept otherwise sections depending
# on it can't be uninstalled # on it can't be uninstalled
collective.recipe.shelloutput collective.recipe.shelloutput
scripts =
gunicorn
supervisord
supervisorctl
[versions] [versions]
Flask-Auth = 0.85 Flask-Auth = 0.85
......
stack/monitor/buildout.cfg
View file @ 4f1be677
...@@ -7,13 +7,13 @@ extends = ...@@ -7,13 +7,13 @@ extends =
../../component/dash/buildout.cfg ../../component/dash/buildout.cfg
../../component/openssl/buildout.cfg ../../component/openssl/buildout.cfg
../../component/lxml-python/buildout.cfg ../../component/lxml-python/buildout.cfg
../../component/pycurl/buildout.cfg
../../component/python-cryptography/buildout.cfg ../../component/python-cryptography/buildout.cfg
../../stack/logrotate/buildout.cfg ../../stack/logrotate/buildout.cfg
../../stack/slapos.cfg
parts = parts =
slapos-cookbook slapos-cookbook
extra-eggs monitor-eggs
monitor2-template monitor2-template
[monitor-download-base] [monitor-download-base]
...@@ -30,25 +30,17 @@ destination = ${buildout:parts-directory}/monitor-scripts ...@@ -30,25 +30,17 @@ destination = ${buildout:parts-directory}/monitor-scripts
on-update = true on-update = true
[monitor-eggs] [monitor-eggs]
# XXX: merge with extra-eggs? recipe = zc.recipe.egg
interpreter = monitor-pythonwitheggs
eggs = eggs =
${lxml-python:egg}
${pycurl:egg}
${python-cryptography:egg}
plone.recipe.command plone.recipe.command
collective.recipe.template collective.recipe.template
cns.recipe.symlink cns.recipe.symlink
slapos.toolbox
slapos.core
[extra-eggs]
recipe = zc.recipe.egg
interpreter = pythonwitheggs
eggs =
${monitor-eggs:eggs}
psutil psutil
PyRSS2Gen PyRSS2Gen
Jinja2 Jinja2
depends =
${slapos-toolbox:recipe}
# Monitor templates files # Monitor templates files
...@@ -82,12 +74,12 @@ rendered = ${buildout:directory}/template-monitor.cfg ...@@ -82,12 +74,12 @@ rendered = ${buildout:directory}/template-monitor.cfg
context = context =
key apache_location apache:location key apache_location apache:location
key template_logrotate_base template-logrotate-base:rendered key template_logrotate_base template-logrotate-base:rendered
raw monitor_bin ${buildout:directory}/bin/monitor.bootstrap raw monitor_bin ${buildout:bin-directory}/monitor.bootstrap
raw monitor_collect ${buildout:directory}/bin/monitor.collect raw monitor_collect ${buildout:bin-directory}/monitor.collect
raw monitor_statistic ${buildout:directory}/bin/monitor.statistic raw monitor_statistic ${buildout:bin-directory}/monitor.statistic
raw monitor_runpromise ${buildout:directory}/bin/monitor.runpromise raw monitor_runpromise ${buildout:bin-directory}/monitor.runpromise
raw monitor_genstatus ${buildout:directory}/bin/monitor.genstatus raw monitor_genstatus ${buildout:bin-directory}/monitor.genstatus
raw monitor_configwrite ${buildout:directory}/bin/monitor.configwrite raw monitor_configwrite ${buildout:bin-directory}/monitor.configwrite
raw monitor_conf_template ${monitor-conf:location}/${monitor-conf:filename} raw monitor_conf_template ${monitor-conf:location}/${monitor-conf:filename}
raw monitor_https_cors ${monitor-httpd-cors:location}/${monitor-httpd-cors:filename} raw monitor_https_cors ${monitor-httpd-cors:location}/${monitor-httpd-cors:filename}
raw curl_executable_location ${curl:location}/bin/curl raw curl_executable_location ${curl:location}/bin/curl
...@@ -97,9 +89,9 @@ context = ...@@ -97,9 +89,9 @@ context =
raw monitor_httpd_template ${monitor-httpd-conf:location}/${monitor-httpd-conf:filename} raw monitor_httpd_template ${monitor-httpd-conf:location}/${monitor-httpd-conf:filename}
raw openssl_executable_location ${openssl:location}/bin/openssl raw openssl_executable_location ${openssl:location}/bin/openssl
raw python_executable ${buildout:executable} raw python_executable ${buildout:executable}
raw python_with_eggs ${buildout:directory}/bin/${extra-eggs:interpreter} raw python_with_eggs ${buildout:bin-directory}/${monitor-eggs:interpreter}
raw template_wrapper ${monitor-template-wrapper:location}/${monitor-template-wrapper:filename} raw template_wrapper ${monitor-template-wrapper:location}/${monitor-template-wrapper:filename}
raw check_disk_space ${buildout:directory}/bin/check-free-disk raw check_disk_space ${buildout:bin-directory}/check-free-disk
raw bin_directory ${buildout:directory}/bin raw bin_directory ${buildout:directory}/bin
[versions] [versions]
......
stack/slapos.cfg
View file @ 4f1be677
...@@ -30,6 +30,8 @@ extends = ...@@ -30,6 +30,8 @@ extends =
../component/python-cachecontrol/buildout.cfg ../component/python-cachecontrol/buildout.cfg
../component/python-cryptography/buildout.cfg ../component/python-cryptography/buildout.cfg
../component/python-PyYAML/buildout.cfg ../component/python-PyYAML/buildout.cfg
../component/pycurl/buildout.cfg
# Separate from site eggs # Separate from site eggs
allowed-eggs-from-site-packages = allowed-eggs-from-site-packages =
...@@ -97,6 +99,22 @@ eggs = ...@@ -97,6 +99,22 @@ eggs =
${python-cryptography:egg} ${python-cryptography:egg}
pyOpenSSL pyOpenSSL
slapos.cookbook slapos.cookbook
# slapos.toolbox containing utilities
[slapos-toolbox]
recipe = zc.recipe.egg
eggs =
${lxml-python:egg}
${pycurl:egg}
${python-cryptography:egg}
slapos.toolbox
# Install a slapos command with networkcache enabled in ${buildout:bin-directory}
[slapos-command]
recipe = zc.recipe.egg
eggs =
${lxml-python:egg}
slapos.core
slapos.libnetworkcache slapos.libnetworkcache
[versions] [versions]
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7