It seems some programs can only be built by the same gcc than the one used to
build golang itself.

For example, when system gcc is gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516,
grafana-server fail to build with:

    /data/slappart11_testnode/cqg/inst/test0-0/tmp/shared/golang1.12/fbee59cfb3c995382cf70d409615aa54/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
    /usr/bin/ld: /tmp/go-link-305363633/000000.o: unable to initialize decompress status for section .debug_info
    /usr/bin/ld: /tmp/go-link-305363633/000000.o: unable to initialize decompress status for section .debug_info
    /usr/bin/ld: /tmp/go-link-305363633/000000.o: unable to initialize decompress status for section .debug_info
    /usr/bin/ld: /tmp/go-link-305363633/000000.o: unable to initialize decompress status for section .debug_info
    /tmp/go-link-305363633/000000.o: file not recognized: File format not recognized
    collect2: error: ld returned 1 exit status

Also generally, if we don't trust system gcc to build golang, we cannot trust
it to build golang programs.

The downside is that if components or software want to use a specifig
golang version they have to set both golang= and the corresponding
gcc-bin-directory= in their [gowork]

/reviewed-on !627

For now we just use a directory inside the partition as shared parts,
but we could also maybe one day also inject the shared_part_list from
this slapos.

This recipe already use all options defined in buildout for substitution
in the erp5testnode.cfg.in

slap_connection is a deprecated alias, we use kebab case in buildout
profiles.

/reviewed-on nexedi/slapos!688

This requires the "prerm" plugin be enabled to be used in addition to devperm. 

I tested on production and it works fine.

/cc @tomo

/reviewed-on nexedi/slapos!687

If there is IPv6 on both interface, Linux will put a default route on
ens3 preventing IPv6 to work correctly on ens4.
Even disabling totally IPv6 from inside the host on ens3 may not work.
It is safer to disable it from qemu process directly. Also, it will ease
the configuration of the host.

NBD server is not running until an image is uploaded. Make sure that the
promise doesn't fail until the image is there. Also display a WARNING in
the connection parameter to say that the NBD server is not running.

Recent fixes for proftpd were still not correct, we still have issues with rpath for mod_auth_web and proftpd itself which uses libcap if available, error was:
    
    ======================================================================
    ERROR: setUpModule (test)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/srv/slappart11/cqg/soft/1ae6b62cded47f49a2e77af2f372cf2a/parts/slapos.core-repository/slapos/testing/testcase.py", line 154, in setUpModule
        installSoftwareUrlList(cls, [software_url], debug=debug)
      File "/srv/slappart11/cqg/soft/1ae6b62cded47f49a2e77af2f372cf2a/parts/slapos.core-repository/slapos/testing/testcase.py", line 348, in installSoftwareUrlList
        raise e
    RuntimeError: /srv/slappart11/cqg/inst/test0-0/tmp/soft/728168be59c5353c6e3c6a6bc4cad052/parts/proftpd/bin/ftpdctl uses system library /lib/x86_64-linux-gnu/libcap.so.2 for libcap.so.2 /srv/slappart11/cqg/inst/test0-0/tmp/soft/728168be59c5353c6e3c6a6bc4cad052/parts/proftpd/libexec/mod_auth_web.so has some not found libraries:
    /srv/slappart11/cqg/inst/test0-0/tmp/soft/728168be59c5353c6e3c6a6bc4cad052/parts/proftpd/libexec/mod_auth_web.so: /usr/lib/x86_64-linux-gnu/libssl.so.1.1: version `OPENSSL_1_1_1' not found (required by /srv/slappart11/cqg/inst/test0-0/tmp/shared/curl/da798e87b4f10345c33fbae0a4593114/lib/libcurl.so.4)
    /srv/slappart11/cqg/inst/test0-0/tmp/soft/728168be59c5353c6e3c6a6bc4cad052/parts/proftpd/sbin/proftpd uses system library /lib/x86_64-linux-gnu/libcap.so.2 for libcap.so.2
    /srv/slappart11/cqg/inst/test0-0/tmp/soft/728168be59c5353c6e3c6a6bc4cad052/parts/proftpd/sbin/in.proftpd uses system library /lib/x86_64-linux-gnu/libcap.so.2 for libcap.so.2
    Ran 0 tests in 530.067s
    ----------------------------------------------------------------------

/reviewed-on nexedi/slapos!685

Because we were simply using "setupClass", all classes where storing
snapshots in the same folder, so if more than one class failed we also
had an error in storing snapshot.

/reviewed-on nexedi/slapos!683

libcap is a system library that some system may have and some may not,
but we don't want slapos software to depend on the base system.

See also d0a88346

This build command never used the LDFLAGS that we set, so instead
rewrite this to inject path as a -R option that will be passed to
libtool.

Now that we check that we don't link with system libraries ( nexedi/slapos.core!172 ) we discovered missing rpaths in our executables.

/reviewed-on nexedi/slapos!682

1.3.6a had a fix for building mod_sftp using OpenSSL 1.1.x, so we can
use default openssl again.
This should fix problems with proftpd-mod_auth_web that was linked
against curl which already used openssl 1.1

We don't seem to need it and we don't want configure script to pickup
system one.

Our software using sshd were sometimes failing in tests, because the way they publish key fingerprint was racy.

It is based on `collective.recipe.shelloutput`, which as we can see in the [recipe code](https://github.com/collective/collective.recipe.shelloutput/blob/78e15c19/collective/recipe/shelloutput/__init__.py) operates on `__init__`.

We are using `collective.recipe.shelloutput` to capture the output of `ssh-keygen -lf $KEY` and this must run after the file `$KEY` is generated ( it is generated by another `plone.recipe.command` version). We were trying to run the `collective.recipe.shelloutput` after the `plone.recipe.command`, but that was incorrect anyway, because `collective.recipe.shelloutput` reads the file at `__init__` step, where `plone.recipe.command` creates the file at `install` step.
As we could see in test suite, it was sometimes working, when `slapos node instance` ran only once, but it sometimes  working, when `slapos node instance` ran more than once, for example because a promise failed and `slapos node instance` was retried.

Since `collective.recipe.shelloutput` does not take into account the exit code of the command but simply capture with `"Error ..."` whatever the command might output on stderr, we add another step checking that the captured output  is not `"Error ..."` and if it is cause a buildout error so that `slapos node instance` is retried and then succeed.

What should happen now is:
 1. `collective.recipe.shelloutput` reads the key fingerprint, the file is not present so it captures `"Error ..."``
 2. a `plone.recipe.command` creates the key
 3. another `plone.recipe.command` checks that the captured fingerprint is not `"Error ..."` it fails
 4. buildout restarts
 5. `collective.recipe.shelloutput` reads key fingerprint correctly.

Slaprunner has been heavily modified, because it was using a `sshkeys_authority` which was incompatible with this as it uses symlinks for keys. Since we don't know what is the purpose of `sshkeys_authority`, we rewrote that software to use simple commands instead of that "ssh keys authority".

/reviewed-on nexedi/slapos!681

slapos.core to 1.5.6 and slapos.toolbox to 0.104:

 * cleanup stale monitor files and stale promises
 * support download-from-binary-cache-force-url-list option

/reviewed-on nexedi/slapos!679

These keys are not managed by trust of a certificate authority, just by
"trust of first use" so it does not make sense to use a key authority.

This also cause difficulties to publish the key fingerprint as a
parameter, because we can't get the key fingerprint until the authority
service is started.

Also enable ecdsa key.

This fixes random failures with slaprunner tests, because the published
fingerprint was never correct on first buildout run.

Existing webrunners will have a new ssh host key after this.