slaprunner: do not symlink everything from software_release/bin

The fixes a conflict with 'runner-exporter' due bugs in cns.recipe.symlink

Anyway, a much better idea would be to add ~/software_release/bin to PATH.

software/slaprunner/buildout.hash.cfg

@@ -18,7 +18,7 @@ md5sum = ec70348dd71b319590a5c5837f3d2e45
 
 [template-runner]
 filename = instance-runner.cfg
-md5sum = b20894f378530e79c847ddfb61782cc5
+md5sum = d383fcdb832031aa4bd00bde94901819
 
 [template-runner-import-script]
 filename = template/runner-import.sh.jinja2
@@ -26,7 +26,7 @@ md5sum = fc22e2d2f03ce58631f157a5b4943e15
 
 [instance-runner-import]
 filename = instance-runner-import.cfg.in
-md5sum = 918fb2984cb2ed7afba9200167f98a0f
+md5sum = 9a3adacb4a833b1af3715e7097d31aad
 
 [instance-runner-export]
 filename = instance-runner-export.cfg.in

software/slaprunner/instance-runner-import.cfg.in

@@ -19,7 +19,6 @@ parts +=
   runtestsuite
   shellinabox
   shellinabox-service
-  symlinks
   slapos-cfg
   cron-entry-prepare-software
   deploy-instance-parameters

software/slaprunner/instance-runner.cfg

@@ -18,7 +18,6 @@ common-runner-parts =
   runner-sshd-promise
   runner-sshd-service
   runtestsuite
-  symlinks
   shellinabox
   shellinabox-service
   slapos-cfg
@@ -43,8 +42,6 @@ common-runner-parts =
 ## Monitoring part
   monitor-base
   monitor-check-webrunner-internal-instance
-## Usability part
-  template-slapuser-script
 
 parts = $${:common-runner-parts}
 extends =
@@ -214,11 +211,10 @@ default_repository_branch = $${slap-parameter:slapos-reference}
 [slaprunner-supervisord-wrapper]
 recipe = slapos.cookbook:wrapper
 # XXX hardcoded locations
-command-line = $${directory:bin}/slapos node supervisord --cfg $${directory:etc}/slapos.cfg -n
+command-line = $${template-slapuser-script:rendered} node supervisord -n
 wrapper-path = $${directory:services}/slaprunner-supervisord
 hash-existing-files = $${buildout:directory}/software_release/buildout.cfg
 
-
 [test-runner]
 <= slaprunner
 slapos.cfg = $${directory:etc}/slapos-test.cfg
@@ -470,7 +466,7 @@ hash-existing-files = $${buildout:directory}/software_release/buildout.cfg
 
 [gunicorn-graceful]
 recipe = slapos.cookbook:wrapper
-command-line = $${directory:bin}/killpidfromfile $${gunicorn:path_pid} SIGHUP
+command-line = ${buildout:directory}/bin/killpidfromfile $${gunicorn:path_pid} SIGHUP
 wrapper-path = $${directory:scripts}/gunicorn-graceful
 
 #--------------------
@@ -634,11 +630,6 @@ name = runner-sshd.py
 config-hostname = $${slap-network-information:global-ipv6}
 config-port = $${runner-sshd-port:port}
 
-[symlinks]
-recipe = cns.recipe.symlink
-symlink_target = $${directory:bin}
-symlink_base = ${buildout:directory}/bin
-
 [slap-parameter]
 # Default value if no ssh key is specified
 user-authorized-key =
@@ -695,7 +686,7 @@ template = inline:
 
 [shellinabox-service]
 recipe = slapos.cookbook:wrapper
-command-line = $${directory:bin}/shellinaboxd
+command-line = $${shellinabox:rendered}
 wrapper-path = $${directory:services}/shellinaboxd
 hash-existing-files = $${buildout:directory}/software_release/buildout.cfg
 

software/slaprunner/software.cfg

@@ -154,7 +154,6 @@ eggs +=
 
 [versions]
 Flask-Auth = 0.85
-cns.recipe.symlink = 0.2.3
 futures = 3.0.5
 gitdb = 0.6.4
 gunicorn = 19.10.0

[Thomas Gambier]

the combination of this + 74e50506 make the section template-slapuser-script not executed anymore so it breaks the resiliency...

/cc @jerome @jm @rafael

@jm could you fix this ?

[Thomas Gambier]

I was not clear enough. In runner1, bin/slapos script doesn't exist. So the importer script is failing because it is using bin/slapos.

[Thomas Gambier]

Fixed in !860 (merged) (27dd12f0)

[Jérome Perrin]

It seems this XXX comment is not longer needed.

[Jérome Perrin]

I wanted to do this, but was afraid that after updating would remove the scripts. Maybe only ~/bin/slapos might have this problem.

[Jérome Perrin]

( if there's actually a problem, I have not tried, maybe it's OK )

[Jérome Perrin]

The part installing ~/bin/slapos is template-slapuser-script, using slapos.recipe.template:jinja2 which was version up not so long time ago, in 7b373b47 . I don't think Vifib used slaprunner in production between 7b373b47 and 1e514399 so that's probably OK.

[Jérome Perrin]

My previous message was not clear ... I assume that if this part has version up it will be reinstalled, so on upgrade, something like this would happen:

• symlinks is uninstalled, it removes ~/bin/slapos
• template-slapuser-script is reinstalled, it restores ~/bin/slapos

[Bryton Lacquement]

Recent slaprunner tests fail and it seems to be related to this commit.

I was able to start tests locally, and we can see early in /srv/slapgrid/slappart2/tmp/slapos/inst/s0/.s0_supervisord-747a0f2ec67058adbe96b2c1b4c8a70d.log:

/srv/slapgrid/slappart2/tmp/slapos/inst/s0/etc/service/supervisord-747a0f2ec67058adbe96b2c1b4c8a70d: 2: exec: /srv/slapgrid/slappart2/tmp/slapos/inst/s0/bin/supervisord: not found
[~10 times]

/srv/slapgrid/slappart2/tmp/slapos/inst/s0/software_release/bin/supervisord exists.

Similar issue for gunicorn.

[Julien Muchembled]

Jérôme, I understood from your first comment.

There are already several reasons that would cause the part to be reinstalled. You mention the version up but there's also the path of the built SR, which changes in most cases (at least everywhere slaprunner is deployed massively).

There's a pending slapos.buildout release that will prevent this kind of bugs. But we have to clean before using it or SR will fail completely.

In the worst case, buildout runs at least once a day and would recreate the ~/bin/slapos if it's missing.

[Julien Muchembled]

Thank you Bryton for notifying.

Previous test results reported a build failure about boost-lib in several SR, e.g. https://nexedi.erp5.net/test_result_module/20200322-7A372355/21: Jérome, do you have an idea?

I wanted to do this, but was afraid that after updating would remove the scripts.

Indeed, I missed a few ones. I'll fix.

[Jérome Perrin]

Previous test results reported a build failure about boost-lib in several SR, e.g. https://nexedi.erp5.net/test_result_module/20200322-7A372355/21: Jérome, do you have an idea?

I had to look a few times :)

RuntimeError: /srv/slapgrid/slappart13/srv/testnode/cvt/inst/test0-0/tmp/shared/boost-lib/19fd51b2d7c2e33aa82dfc86c196153b__compile__/boost_1_67_0/bin.v2/libs/type_erasure/build/gcc-5.5.0/release/threadapi-pthread/threading-multi/libboost_type_erasure.so.1.67.0 has

This is checking in __compile__ dir. Apparently build failed for one software release test, then next other software release tests from the same test run are finding errors in this __compile__ dir. This /srv/slapgrid/slappart13/srv/testnode/cvt/inst/test0-0/tmp is erased at the beginning of each test, so we can probably ignore. If this happens often we can probably had an exception to ignore */*__compile__, that's the first time I see it.

[Jérome Perrin]

If this happens often we can probably had an exception to ignore */*__compile__, that's the first time I see it.

Done as part of slapos.core!202 (merged)