helloweb/ruby: Abandon usage of slapos.cookbook:wrapper
It turned out we cannot currently use slapos.cookbook in software part of SR - the reason is slapos.cookbook egg depends on lxml egg, which in turn needs libxml to be also installed via slapos. For this reason stack/slapos.cfg has [slapos-cookbook] # NOTE _not_ slapos.cookbook recipe = zc.recipe.egg eggs = ${lxml-python:egg} ... and lxml-python is lxml building recipe to build it together with libxml: [lxml-python] recipe = zc.recipe.egg:custom egg = lxml rpath = ${libxml2:location}/lib/ ${libxslt:location}/lib/ ${zlib:location}/lib/ environment = lxml-python-env ... So underlying idea, as I understand it, is: every SR contains slapos-cookbook in parts and this way lxml-python gets build. Then when there is slapos.cookbook egg usage, it is already correctly built. BUT This works when such slapos.cookbook egg usage happens _only_ in instance part of an SR: otherwise, if buildout sees slapos.cookbook egg usage in some recipe, e.g. like it currently is in helloweb-ruby: [helloweb-ruby] recipe = slapos.cookbook:wrapper ... it _first_ tries to install slapos.cookbook egg directly - as needed for recipes eggs are installed as a first step, _before_ further buildout processing. What happens then is that slapos.cookbook (note not "-") egg sources and dependencies are downloaded from pypi, including lxml egg, all are tried to build, and in lxml egg it fails this way: ... Processing lxml-3.5.0 Writing /tmp/tmpLGK4xWbuild/lxml-3.5.0/setup.cfg Running setup.py -q bdist_egg --dist-dir /tmp/tmpLGK4xWbuild/lxml-3.5.0/egg-dist-tmp-DJvofa Building lxml version 3.5.0. Building without Cython. ERROR: /bin/sh: 1: xslt-config: not found ** make sure the development packages of libxml2 and libxslt are installed ** Using build configuration of libxslt In file included from src/lxml/lxml.etree.c:323:0: src/lxml/includes/etree_defs.h:14:31: fatal error: libxml/xmlversion.h: No such file or directory #include "libxml/xmlversion.h" ^ compilation terminated. Compile failed: command 'gcc' failed with exit status 1 /tmp/tmpLGK4xWbuild/lxml-3.5.0/temp/xmlXPathInitGlEAOF.c:1:26: fatal error: libxml/xpath.h: No such file or directory #include "libxml/xpath.h" ^ compilation terminated. ********************************************************************************* Could not find function xmlCheckVersion in library libxml2. Is libxml2 installed? ********************************************************************************* error: Setup script exited with error: command 'gcc' failed with exit status 1 An error occurred when trying to install lxml 3.5.0. Look above this message for any errors that were output by easy_install. Could't load zc.buildout entry point wrapper from slapos.cookbook: Couldn't install: lxml 3.5.0. While: Installing. Getting section helloweb-ruby. Initializing section helloweb-ruby. Installing recipe slapos.cookbook. Error: Couldn't install: lxml 3.5.0 Previously it probably used to work because we had system libxml installed, and this way lxml compilation succeeded (but was incorrect from slapos point of view). ( The problem turned out to be already known somehow - see e.g. c7d00913 "Initial neoppod commit." and look for "Note on LXML/END LXML" there ) Solution could be: either fix slapos.cookbook installation via e.g. teaching buildout to take into account pre-dependencies for eggs (for lxml) or just to avoid using slapos.cookbook:wrapper for executable generation. While @kazuhiko is working on the first more-generic solution, here goes a simpler one to just make helloweb component alive again: like it is done in a lot of places (e.g. in software/kvm/) let's use collective.recipe.template to generate a short shell script. NOTE previously the command line was ${bundler:bundle} exec sh -c 'helloweb.rb "$@"' ${:_buildout_section_name_} but now it is the same with "$@" appended: exec ${bundler:bundle} exec sh -c 'helloweb.rb "$@"' ${:_buildout_section_name_} "$@" The reason is slapos.cookbook:wrapper uses slapos.recipe.librecipe.execute.generic_exec() in generated script, which appends sys.argv[1:] to the command-line implicitly: def generic_exec(args): ... os.execve(exec_list[0], exec_list + sys.argv[1:], exec_env) ^^^^^^^^^^^^^^ https://lab.nexedi.com/nexedi/slapos/blob/54bbe0a9/slapos/recipe/librecipe/execute.py#L84 that's why last "$@" was not present in original version. P.S. Otherwise currently slapos.cookbook is used only in instance parts of recipes in whole slapos.git /reviewed-by TrustMe /debugged-with @kazuhiko /cc @vpelletier
Showing