Features:

 * forcediphttpsadapter for SNI
 * compatbile with apache-frontend SR
 * local server to test against
 * supervisor state checks
 * promise list checks
 * test data (to save/load textual assertions)
 * own root CA for HTTPs backend

Based on Jerome's work from /software/proftpd/test/utils.py

Main changes:

 * tearDownClass is called in case of errors in setUpClass (mimic addCleanup
   instance method)
 * the setUpClass is split to smaller methods for ease of further development
 * full instance path is exposed for test introspection
 * software path of master partition is exposed for test introspection

proxy.timeout is provided

In order to follow proper way, move caddy to the component and reuse it.

  The webrunner user don't necessary uses webrunner to develop slapos,
  he can use webrunner to develop a new component, where instance is never
  created.

  If the software release don't instantiate instances, skip from continue and consider
  the restauration job done (data is copied, software is built).

 Once the requests to cloudooo software types were removed,
 orphan instances might be kept arround due a lack of implementation
 of garbage collection on slapos proxy.

 This change allows the ghost cloudooo instances, don't fail, until
 the garbage collection implementation be deployed, reducing the
 number of irrelevant failures on resilience after an upgrade.

Requirement for nexedi/erp5!695

/reviewed-on nexedi/slapos!343

jinja2 template is required, as some paramters might come from the network.

/reviewed-on nexedi/slapos!345

/reviewed-on nexedi/slapos!346

Filename is split as an URL path, and joined in OS-dependent manner.
Remove corresponding comment in all buildout.hash.cfg files.

This prevent outdated instances to fail resilience, so it makes this change backward compatible.

Do not upper filenames nor users, use them as is.

/reviewed-on !339

No custom_http nor custom_https shall publish information normally calculated.

/reviewed-on nexedi/slapos!340

This brings the following changes of git-backup:

	kirr/git-backup@0b8d834b...cc6ac54f

the most important of which is dramatical speedup of pull and 30-50%
speedup of restore if bitmap index of backup repo is built and git is
recent enough.

Please see the following commits for details:

	kirr/git-backup@899103bf
	kirr/git-backup@3efed898
	kirr/git-backup@0ab7bbb6

/cc @alain.takoudjou, @rafael, @jerome

/reviewed-on nexedi/slapos!342

Extended KVM SR does not have this hosts in allowed, and we need to stick
with it.

For reference in case if some existing Python distribution does not install,
it was detected with "git diff 8deeab.. -- stack/slapos.cfg"

The Debian packaging of our RINA kernel builds with GCC 4.9,
which is not available on Stretch:

  # apt install ./linux-build-deps_4.1.33+rina65.g2e1cdbcb4-1_amd64.deb
  ...
  The following packages have unmet dependencies:
  linux-build-deps : Depends: gcc-4.9 but it is not installable

The regexp was matching "slappart10" in `/path/to/slappart10/software.cfg`,
was also matching "slappart10" in `/path/to/aaslappart10aa/software.cfg`
and was not matching "slappart1" in `/path/to/slappart1/software.cfg`
or "slappart1000" in `/path/to/slappart1000/software.cfg`.

The fix allows to match "slappartX" in `/path/to/slappartX/software.cfg`
where X can be between 0 to infinite. Also, it protects from matching
"slappartX" in `/path/to/aaaslappartXaaa/software.cfg`.

Now that the test suite passes again for Jessie, let's try Stretch.

If two NayuOS images are built on the same day, the second one will not get zipped. The -f flag in gzip can be used to overwrite the previous image

/reviewed-on nexedi/slapos!336

This SR was developed and tested in environments where both software and instance where owned by the same unix user. These are fixes for problems discovered when running on a "system level" slapos.

This cannot easily be tested without actually using different users (especially the mistake on the mode of the executable script), but this MR introduce some tests to check that service running in instance does not open files or unix sockets outside of instance. It also introduce a utility method to access the supervisor RPC API, it can be useful if tests needs to start/stop a process or get process PID to inspect this process.

I ran into problems, because proftpd sets [`PR_SET_DUMPABLE`](http://man7.org/linux/man-pages/man2/prctl.2.html) to 0, which disables any possibility of inspecting a running program. See 18e28bbb  for details about the chosen approach.

/cc @rafael @Nicolas @luke @Eteri @vpelletier 

/reviewed-on nexedi/slapos!334

erp5, mariadb: Allow users to set additional environment variables to mysqld to use third party ODBC libraries for CONNECT storage engine.

more discussion at nexedi/slapos!266

an utility method to control and query supervisor instance used by
slapos using http://supervisord.org/api.html

Unless compiled with --enable-devel=coredump, proftpd's mod_rlimit will
use a prctl(PR_SET_DUMPABLE, 0) syscall[1] which prevents us from
inspecting files open by this process[2]

We don't want to enable debug build and I did not find a way to disable
only this PR_SET_DUMPABLE flag setting.

[1] https://github.com/proftpd/proftpd/blob/v1.3.6/modules/mod_rlimit.c#L580-L583
[2] http://man7.org/linux/man-pages/man5/proc.5.html

Some files and sockets where created in their default location, which
was in the software.
As a result, this software was working only on webrunner or testnodes,
which uses same unix user for software and instance.

To be usable in a "system level" slapos with multiple users,
executables scripts must be 0755

!311 was merged a bit too fast. After test suite was repaired ( the test node was stuck updating the git repository ), [test](https://nexedi.erp5.net/test_result_module/20180507-19395E1F/2) was not passing on testnode, because path was too deep.

While debugging this, I realized this check was wrong, because supervisor [also append the pid](https://github.com/Supervisor/supervisor/blob/70720a0311d868fbf58a31cd26a7147921dcff29/supervisor/http.py#L564-L574) to this path, so the path is even longer. I just checked that we have room for 7 digit pids.

Because path was too long on test node, test now run with `SLAPOS_TEST_WORKING_DIR` set to a slightly shorter path that's enough for this case.

There are some other minor fixes, see individual commits.

/cc @Nicolas @luke @rafael 

/reviewed-on nexedi/slapos!325