See merge request nexedi/slapos!839

https://github.com/eclipse-theia/theia/blob/v1.7.0/CHANGELOG.md#v170---29102020

Future Cython+ work will use it.

Fix for #20200514-218C705 - \[testnode\] frontend for log access

Depends on nexedi/erp5!1304

See merge request nexedi/slapos!848

This fixes the nextcloud SR, which was broken by
commit 92779bf4
(mariadb is not a part anymore).

This fixes commit a62e5e7b.
See also commit 491e6e28.

And set it as log_frontend_url in testnode config

Just add the following 2 lines in a SR:

[mariadb]
location = ${mariadb-10.4:location}

It does not build with GCC 8.2

Going Go1.14.9 -> Go1.14.10 brings in compiler and runtime fixes
including fix for crash in garbage-collector due to race condition:

https://github.com/golang/go/issues/40642
https://github.com/golang/go/issues/40641

Tested on helloworld SR.

See merge request nexedi/slapos!844

validators.url is enough, even for Caddy, to check that URL is correct, and
caddy_backend_url_validator was introduced before validators.

Also calling an external command for each slave takes a lot of time.

Thanks to this other sections can directly reference them, and so they are
correctly created as needed, so linking section does not need update-command

The password is anyway present in the section itself, so it's eventual change
will result with reinstalling the section.

Also change a bit existing frontend_url to manage it the same way.

This way buildout can reuse egg caches and it's a bit faster:

To run a simple instance buildout, from 2.837s it goes down to 1.875s.
To run slapos node instance 10 times just after requesting an ERP5 instance, it goes from ~112s to 98s

before:

    hyperfine "/srv/slapgrid/slappart4/srv/slapos/inst/slappart0/tmp/shared/python2.7/60364a13cc977dd5a894e0239ac889b9/bin/python2.7 /srv/slapgrid/slappart4/srv/slapos/inst/slappart0/tmp/soft/c63ba7265399450b28f9ea6d5667a5e7/bin/buildout -U"
    Benchmark #1: /srv/slapgrid/slappart4/srv/slapos/inst/slappart0/tmp/shared/python2.7/60364a13cc977dd5a894e0239ac889b9/bin/python2.7 /srv/slapgrid/slappart4/srv/slapos/inst/slappart0/tmp/soft/c63ba7265399450b28f9ea6d5667a5e7/bin/buildout -U
      Time (mean ± σ):      2.837 s ±  0.275 s    [User: 2.481 s, System: 0.285 s]
      Range (min … max):    2.482 s …  3.222 s    10 runs

after:

    hyperfine "/srv/slapgrid/slappart4/srv/slapos/inst/slappart0/tmp/shared/python2.7/60364a13cc977dd5a894e0239ac889b9/bin/python2.7 /srv/slapgrid/slappart4/srv/slapos/inst/slappart0/tmp/soft/c63ba7265399450b28f9ea6d5667a5e7/bin/buildout -U"
    Benchmark #1: /srv/slapgrid/slappart4/srv/slapos/inst/slappart0/tmp/shared/python2.7/60364a13cc977dd5a894e0239ac889b9/bin/python2.7 /srv/slapgrid/slappart4/srv/slapos/inst/slappart0/tmp/soft/c63ba7265399450b28f9ea6d5667a5e7/bin/buildout -U
      Time (mean ± σ):      1.875 s ±  0.067 s    [User: 1.660 s, System: 0.148 s]
      Range (min … max):    1.816 s …  2.038 s    10 runs

See merge request nexedi/slapos!846

Changes configuration files to run repman tests in python3.

Adds the newly added to nexedi's repositories rubygemsrecipe (https://lab.nexedi.com/nexedi/rubygemsrecipe) to the list of tested eggs.

This also fixes rpath of rust binaries.

See merge request nexedi/slapos!845

When proftpd software release is tested locally, the socket named /srv/slapgrid/slappart76/srv/runner/instance/slappart1/tmp/soft/91d420e3970a2088e648d2eb86e155ea/parts/prof is created but never removed.
First it is not an appropriated directory to create a socket and then not removing this socket leads to an error if tests are run a second time :

subprocess.CalledProcessError: Command '('ldd', '/srv/slapgrid/slappart76/srv/runner/instance/sla
ppart1/tmp/soft/91d420e3970a2088e648d2eb86e155ea/parts/prof')' returned non-zero exit status 1.

----------------------------------------------------------------------
Ran 0 tests in 9.914s

FAILED (errors=1)

This is due to code in https://github.com/proftpd/proftpd/blob/master/src/ctrls.c :

const char *socket_path = PR_RUN_DIR "/test.sock"; // socket_path="/srv/slapgrid/slappart76/srv/runner/instance/slappart1/tmp/soft/91d420e3970a2088e648d2eb86e155ea/parts/proftp/var/test.sock"
sstrncpy(sockun.sun_path, socket_path, sizeof(sockun.sun_path)); // sockun.sun_path="/srv/slapgrid/slappart76/srv/runner/instance/slappart1/tmp/soft/91d420e3970a2088e648d2eb86e155ea/parts/prof"

where `sun_path` is limited to UNIX_PATH_MAX (108 characters): char sun_path[UNIX_PATH_MAX]; https://elixir.bootlin.com/linux/latest/source/include/uapi/linux/un.h#L9

See merge request nexedi/slapos!838

We don't plan using varnish like this.

Move `numpy=1.16.4` from all over the place into component/numpy.
Don't move if a different numpy version is used, or it looks like a
software cares to use exactly particular version.
Downgrade pygolang/test.cfg from numpy=1.16.6 to numpy=1.16.4 and use
common numpy component version - using numpy=1.16.6 is not required for
pygolang testing and so this downgrade is acceptable. It will be better
to upgrade NumPy to latest in component/numpy/ as a future separate step.

See previous patch where it was decided and explained that version for
component <X> lives in component/X/.

Move versions for eggs that component/{ZEO,pygolang,zodbtools,pytest}
depend on out of component/ and into stack/slapos.cfg

Leave version of component <X> inside component/<X>.

I was asked to do so:

!839 (comment 119170)

I tried to use this gdb in a software release installed by slapos-sr-testing (which perform some extra checks that we only do in slapos-sr-testing at the moment) and it complains that:

```
======================================================================
ERROR: setUpModule (test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/srv/slapgrid/slappart9/srv/slapos/soft/24930952d96110d7e0142b49eba918a8/parts/slapos.core-repository/slapos/testing/testcase.py", line 168, in setUpModule
    installSoftwareUrlList(cls, [software_url], debug=debug)
  File "/srv/slapgrid/slappart9/srv/slapos/soft/24930952d96110d7e0142b49eba918a8/parts/slapos.core-repository/slapos/testing/testcase.py", line 378, in installSoftwareUrlList
    checkSoftware(cls.slap, software_url)
  File "/srv/slapgrid/slappart9/srv/slapos/soft/24930952d96110d7e0142b49eba918a8/parts/slapos.core-repository/slapos/testing/testcase.py", line 336, in checkSoftware
    raise RuntimeError('\n'.join(error_list))
RuntimeError: /srv/slapgrid/slappart9/srv/slapos/inst/slappart7/tmp/shared/gdb/d14016b094c3637c4ebf6a3df4a8c64d/bin/gdb uses system library /usr/lib/x86_64-linux-gnu/libexpat.so.1.6.8 for libexpat.so.1

----------------------------------------------------------------------
Ran 0 tests in 462.686s
```

I saw this on a debian KVM on which I don't think I installed any system package except `slapos-node`, but `libexpat1-dev` is installed for some reason, so configure detected it and the built binary is linked against it. I checked some test nodes and `/usr/include/expat.h` seems present on some testnodes, so probably some test nodes will also have this problem. For that, I suggest to build a gdb using slapos `libexpat`, so that we build something more reproductible and not depending on the host system (that's important if we want to use binary cache, but can also be a problem if system packages get remove/upgraded). As you might be thinking now, that's a kind of infinite problem, because we can not add anything that configure might detect, but that's how we usually do for this for now.

While looking at this, I also realised that we were using a .tar.xz URL here, for which rely our recipe will use `xz` command. Because some machines might not have that command, we usually put `xz-utils` in PATH.

How about including something like this ?

Just like with pygolang and zodbtools add way to run wendelin.core tests
via nxdtest with test instance organized with help of stack/nxdtest.cfg

Test agains ZODB4 and ZODB5 for the reasons explained in previous patch.

Wendelin.core already had [wendelin.core-dev], so we only have to add
test*.cfg in this patch.

Following approach used for pygolang in the previous patch lets add
testing support for zodbtools:

- Add zodbtools/buildout-dev.cfg that overrides [zodbtools] to use the
  software from git checkout.

- Add zodbtools/test<X>.cfg that is software-release to create a test
  instance to be run under testnode. To help itself on this task this
  software release uses just-added stack/nxdtest.cfg and
  [python-interpreter] from pygolang, so the code in zodbtools is
  minimal.

Zodbtools can be tested against both ZODB4 and ZODB5 because we still
use ZODB4 as our primary ZODB version, but it is no longer supported by
upstream which considers only ZODB5 as current.

- Add pygolang/buildout-dev.cfg that overrides [pygolang] to use the
  software from git checkout.

- Add pygolang/test.cfg that is software-release to create a test
  instance to be run under testnode. This software release uses
  just-added stack/nxdtest.cfg to help itself on this task, so the code
  in pygolang is minimal.

A new section [python-interpreter] is added, because python interpreters
that zc.recipe.egg generates don't process `-m args` correctly and
handle subprocess well. I had to workaround that with code from
gpython.pymain to be able to run `python -m pytest --<pytestarg>` and to
spawn children processes with preserving sys.path. Comments around and
inside [python-interpreter] has more details on this topic.

Pytest is used to test wendelin.core, zodbtools and pygolang
It is also used in component/unstable/pyodide, but currently via pip.