See merge request nexedi/slapos.package!144

33.1.0 fails to find zc.buildout

This replaces a long pipeline that only failed if the last pipe
command fails. But on Arch, we got the following error:

  xargs: file: terminated by signal 31

and the build continued, pretending it was successful whereas the
package was unusable.

This is similar to bash's pipefail.

Note that 'chrpath -k' exit code is ignored
because it is expected to return 2.

Fedora distribution refuses shebang with unknown python version:
ERROR: ambiguous python shebang in /opt/slapos/tmp-networkcached/bin/networkcache-download: #!/usr/bin/python. Change it to python3 (or python2) explicitly.
ERROR: ambiguous python shebang in /opt/slapos/tmp-networkcached/bin/networkcache-upload: #!/usr/bin/python. Change it to python3 (or python2) explicitly.
ERROR: ambiguous python shebang in /opt/slapos/tmp-networkcached/bin/buildout: #!/usr/bin/python. Change it to python3 (or python2) explicitly.
ERROR: ambiguous python shebang in /opt/slapos/tmp-networkcached/bin/generate-signature-key: #!/usr/bin/python. Change it to python3 (or python2) explicitly.

Using python2.7 explicitely will make the shebang /usr/bin/python2.7

This is needed specially for kernel after 4.16 (especially 4.19, kernel
version of Debian 10) because much more objects are counted in this
max_size limit.

See interesting discussion here: https://lore.kernel.org/patchwork/patch/1245043/

Also stop using deprecated TarFile.add's exclude parameter.

Shipping binaries in /usr/sbin fails because /usr/sbin is a symlink
to /usr/bin. Similarly, /lib is a symlink to /usr/lib.

https://wiki.archlinux.org/index.php/Frequently_asked_questions#Does_Arch_follow_the_Linux_Foundation's_Filesystem_Hierarchy_Standard_(FHS)?

Unfortunately, pip download is no good at downloading only. It always want to instal the egg which is not feasible (see this issue https://github.com/pypa/pip/issues/1884).

Fixup for 6e5b4d76

also remove slapos-test-node playbook as we don't need it anymore

See merge request nexedi/slapos.package!136

See merge request nexedi/slapos.package!135

In addition to linux-image we have to explicitly install/upgrade kernel
headers as well because once they are installed via nxd-fuse-dkms ->
dkms -> linux-headers-amd64, they won't be upgraded, and if
linux-image-amd64 gets upgraded, but headers are not, nxd-fuse-dkms will
be skipped to recompile for upgraded kernel.

!133 (comment 121068)

See merge request !133

Debian 9 and Debian10 ship linux-4.9.x and linux-4.19.x correspondingly.
However kernel packages on those distributions include ABINAME as
package version, for example linux-image-4.9.0-13-amd64
means linux 4.9.x with ABINAME=13.

The ABINAME is there because Linux sometimes breaks ABI compatibility in
small places and Debian is very strict on not throwing ABI changes onto
users unless requested.  However even with that protection Debian
"strongly recommends" to explicitly install just linux-image-amd64 -
without ABINAME - which is just a dependency package that depends on the
latest kernel in particular distribution series:

https://wiki.debian.org/DebianKernelABIChanges

The reason for this recommendation is that upstream Linux rarely breaks
any ABI and even if there is a breakage it is so small and in obscure
places that in practice it does not affect people.

Today, for Debian9, we are explicitly requesting to install Linux 4.9
with ABINAME=13. That stops on 4.9.228 while current Linux 4.9 is
4.9.240 bumped to ABI 14. Before ABI=13, we were requesting to install
ABI=11 which stopped on 4.9.189 and was v↑'ed in 1f249bf7 (playbook:
debian9: kernel v↑  (4.9.189-3 -> 4.9.228-1). In other words by
explicitly specifying linux ABINAME we prevent to keep on updating Linux
kernel to latest _stable_ updates provided by upstream Linux and the
distribution.

Another inconvenience of installing Linux with explicit ABINAME is
interaction with nxd-fuse.ko: this module comes with nxd-fuse-dkms
package, which uses dkms to build itself, and dkms recommends
linux-headers-amd64 for modules that it manages to be able to build.

However we recently saw an issue when linux-headers-amd64 was installed
as latest and depending on linux-headers-14-amd64 (NOTE ABINAME=14),
while the kernel on that server was only linux-image-4.9.0-13-amd64
(NOTE ABINAME=13) as requested by our playbook. As the result nxd-fuse
was skipped to compile, failed to be loaded and FUSE became non-working
on that machine:

    !132 (comment 120438)

    root@rapidspace-testnode-005:~# apt install nxd-fuse-dkms
    Reading package lists... Done
    Building dependency tree
    Reading state information... Done
    The following NEW packages will be installed:
      nxd-fuse-dkms
    0 upgraded, 1 newly installed, 0 to remove and 105 not upgraded.
    Need to get 0 B/58.8 kB of archives.
    After this operation, 295 kB of additional disk space will be used.
    Selecting previously unselected package nxd-fuse-dkms.
    (Reading database ... 80295 files and directories currently installed.)
    Preparing to unpack .../nxd-fuse-dkms_4.9.nxd3+debian2_all.deb ...
    Unpacking nxd-fuse-dkms (4.9.nxd3+debian2) ...
    Setting up nxd-fuse-dkms (4.9.nxd3+debian2) ...
    Loading new nxd-fuse-4.9.nxd3+debian2 DKMS files...
    Building for 4.9.0-13-amd64
    Module build for kernel 4.9.0-13-amd64 was skipped since the
    kernel headers for this kernel does not seem to be installed.

-> Fix it by requiring only linux-image-amd64 without specifying ABINAME and
relying on upstream Linux and distro to provide stable updates for the kernel.

See merge request nexedi/slapos.package!134

See merge request nexedi/slapos.package!132

See merge request nexedi/slapos.package!131

See merge request nexedi/slapos.package!130