Currently Nexedi takes mostly a copy-paste approach when it comes to
organizing bin/runTestSuite and an instance to run this inside testnode:

    kirr@deco:~/src/wendelin/slapos/slapos-master$ git ls-files |grep runTestSuite
    software/build-rina/runTestSuite.in
    software/buildout-testing/runTestSuite.in
    software/cython-test/runTestSuite.in
    software/erp5testnode/testsuite/deploy-test/runTestSuite.py
    software/jstestnode/runTestSuite.in
    software/neoppod/runTestSuite.in
    software/neoppod/stress-testing/runTestSuite.in
    software/unstable/pyodide/runTestSuite.in

and adding new tests to be run this way will only increase the entropy.

Let's stop this, at least for new tests, and use a uniform approach without copying:

1) I've created nxdtest tool[1] which allows a project to specify in
   declarative way how to test itself: given such declaration nxdtest
   takes on itself the responsibility to communicate with testnode
   master, to drive test commands spawning etc, and leaves to the
   project only semantic part that is specific to the particular
   project.

   Nxdtest is based on my go/neo/t/nxd/runTestSuite[2,3] and on the code
   I've seen elsewhere scattered in other runTestSuite.in files.

2) Let's add stack/nxdtest.cfg to SlapOS that provides easy way for a
   software or component to organize testing of itself.

This patch adds stack/nxdtest and migrates software/neotest to use it
(because stack/nxdtest is based on it and this way it helps to track the
changes).

In the following patches we will add testing support for pygolang,
zodbtools and wendelin.core .

[1] https://lab.nexedi.com/kirr/nxdtest/blob/master/nxdtest
[2] kirr/neo@51b18490
[3] kirr/neo@f67c147d

${go:exe} is standalone executable that runs go in activated gowork environment.
It is handy to use in recipes where one cannot source ${gowork:env} -
in particular parts with recipe=zc.recipe.egg

Wendelin.core 2 will use `GO=${go:exe}` in its build environment.

GDB is used in wendelin.core tests:
https://lab.nexedi.com/nexedi/wendelin.core/blob/v0.13-49-g35cb144/t/tfault-run#L28

Starting from 0.14 (aka 2) wendelin.core requires pygolang to be present
both at runtime and at build time.

Update lab.nexedi.com/kirr/neo.git and lab.nexedi.com/kirr/go123 to latest.
Update all other go packages.

The update puls in the following new dependencies:

    github.com/shamaton/msgpack
    github.com/tinylib/msgp
      github.com/philhofer/fwd
      github.com/ttacon/chalk
      golang.org/x/mod
      golang.org/x/xerrors

    github.com/davecgh/go-spew
    github.com/pmezard/go-difflib
    gopkg.in/yaml.v3

Going from 0.0.0.dev4 to -> 0.0.0.dev8 zodbtools:

- Stabilized `zodb dump` format and layed ground for `zodb restore`;
- Taught `zodb analyze` to work with any ZODB storage (instead of being
  FileStorage-only tool) and to analyze a particular range of history
  (instead of crunching data for days on a large database);
- Added `zodb commit` tool that is handy in testing;
- Added ability to specify tid ranges in human-readable format, as in
  e.g. `zodb analyze data.fs 2018-01-01T10:30:00Z..yesterday`.
- Progressed on Python3 support.

See https://pypi.org/project/zodbtools/#zodbtools-change-history
for details.

Close to non-functional change, but removes PendingDeprecationWarning
about cgi.parse_qsl

0.0.0.dev4 to 0.0.7.post1 goes a long way. See
https://pypi.org/project/pygolang/#pygolang-change-history for details.

Recent pygolang is needed for wendelin.core 2.
Zodbtools also uses it starting from v0.0.0.dev5.

We already patch ZEO4 with TCP_NODELAY patch (see 5cf4cf1f "ERP5:
enable TCP_NODELAY for ZEO") and we will need to backport more patches
to ZEO4 branch for wendelin.core 2 to work correctly.

It's not only software/neoppod which uses ZEO, and it is not convenient for
all other software-releases to inherit from neoppod to use correct
version and build of ZEO egg. For this reason factor out details of ZEO
egg building into component/ZEO and let users use ${ZEO:egg} where ZEO
is needed. This way ZEO will be correctly installed for all users.

This patch should be a non-functional change. We switch to
nexedi/ZEO@5114f909 revision which corresponds to ZEO 4.3.1 +
TCP_NODELAY.patch

Adding other patches to ZEO4 needed by wendelin.core 2 will be done as a
separate step.

https://github.com/eclipse-theia/theia/blob/v1.7.0/CHANGELOG.md#v170---29102020

Future Cython+ work will use it.

Fix for #20200514-218C705 - \[testnode\] frontend for log access

Depends on nexedi/erp5!1304

See merge request nexedi/slapos!848

This fixes the nextcloud SR, which was broken by
commit 92779bf4
(mariadb is not a part anymore).

This fixes commit a62e5e7b.
See also commit 491e6e28.

And set it as log_frontend_url in testnode config

Just add the following 2 lines in a SR:

[mariadb]
location = ${mariadb-10.4:location}

It does not build with GCC 8.2

Going Go1.14.9 -> Go1.14.10 brings in compiler and runtime fixes
including fix for crash in garbage-collector due to race condition:

https://github.com/golang/go/issues/40642
https://github.com/golang/go/issues/40641

Tested on helloworld SR.

See merge request nexedi/slapos!844

validators.url is enough, even for Caddy, to check that URL is correct, and
caddy_backend_url_validator was introduced before validators.

Also calling an external command for each slave takes a lot of time.

Thanks to this other sections can directly reference them, and so they are
correctly created as needed, so linking section does not need update-command

The password is anyway present in the section itself, so it's eventual change
will result with reinstalling the section.

Also change a bit existing frontend_url to manage it the same way.

This way buildout can reuse egg caches and it's a bit faster:

To run a simple instance buildout, from 2.837s it goes down to 1.875s.
To run slapos node instance 10 times just after requesting an ERP5 instance, it goes from ~112s to 98s

before:

    hyperfine "/srv/slapgrid/slappart4/srv/slapos/inst/slappart0/tmp/shared/python2.7/60364a13cc977dd5a894e0239ac889b9/bin/python2.7 /srv/slapgrid/slappart4/srv/slapos/inst/slappart0/tmp/soft/c63ba7265399450b28f9ea6d5667a5e7/bin/buildout -U"
    Benchmark #1: /srv/slapgrid/slappart4/srv/slapos/inst/slappart0/tmp/shared/python2.7/60364a13cc977dd5a894e0239ac889b9/bin/python2.7 /srv/slapgrid/slappart4/srv/slapos/inst/slappart0/tmp/soft/c63ba7265399450b28f9ea6d5667a5e7/bin/buildout -U
      Time (mean ± σ):      2.837 s ±  0.275 s    [User: 2.481 s, System: 0.285 s]
      Range (min … max):    2.482 s …  3.222 s    10 runs

after:

    hyperfine "/srv/slapgrid/slappart4/srv/slapos/inst/slappart0/tmp/shared/python2.7/60364a13cc977dd5a894e0239ac889b9/bin/python2.7 /srv/slapgrid/slappart4/srv/slapos/inst/slappart0/tmp/soft/c63ba7265399450b28f9ea6d5667a5e7/bin/buildout -U"
    Benchmark #1: /srv/slapgrid/slappart4/srv/slapos/inst/slappart0/tmp/shared/python2.7/60364a13cc977dd5a894e0239ac889b9/bin/python2.7 /srv/slapgrid/slappart4/srv/slapos/inst/slappart0/tmp/soft/c63ba7265399450b28f9ea6d5667a5e7/bin/buildout -U
      Time (mean ± σ):      1.875 s ±  0.067 s    [User: 1.660 s, System: 0.148 s]
      Range (min … max):    1.816 s …  2.038 s    10 runs

See merge request nexedi/slapos!846

Changes configuration files to run repman tests in python3.

Adds the newly added to nexedi's repositories rubygemsrecipe (https://lab.nexedi.com/nexedi/rubygemsrecipe) to the list of tested eggs.

This also fixes rpath of rust binaries.

See merge request nexedi/slapos!845

When proftpd software release is tested locally, the socket named /srv/slapgrid/slappart76/srv/runner/instance/slappart1/tmp/soft/91d420e3970a2088e648d2eb86e155ea/parts/prof is created but never removed.
First it is not an appropriated directory to create a socket and then not removing this socket leads to an error if tests are run a second time :

subprocess.CalledProcessError: Command '('ldd', '/srv/slapgrid/slappart76/srv/runner/instance/sla
ppart1/tmp/soft/91d420e3970a2088e648d2eb86e155ea/parts/prof')' returned non-zero exit status 1.

----------------------------------------------------------------------
Ran 0 tests in 9.914s

FAILED (errors=1)

This is due to code in https://github.com/proftpd/proftpd/blob/master/src/ctrls.c :

const char *socket_path = PR_RUN_DIR "/test.sock"; // socket_path="/srv/slapgrid/slappart76/srv/runner/instance/slappart1/tmp/soft/91d420e3970a2088e648d2eb86e155ea/parts/proftp/var/test.sock"
sstrncpy(sockun.sun_path, socket_path, sizeof(sockun.sun_path)); // sockun.sun_path="/srv/slapgrid/slappart76/srv/runner/instance/slappart1/tmp/soft/91d420e3970a2088e648d2eb86e155ea/parts/prof"

where `sun_path` is limited to UNIX_PATH_MAX (108 characters): char sun_path[UNIX_PATH_MAX]; https://elixir.bootlin.com/linux/latest/source/include/uapi/linux/un.h#L9