AIKC - Automatic Internal Kedifa's Caucase CSR signing, which can be triggered
by option automatic-internal-kedifa-caucase-csr.

It signs all CSR which match csr_id and certificate from the nodes which needs them.

csr_id is exposed over HTTPS with short living self signed certificate,
which is transmitted via SlapOS Master. Thanks to this, it is possible to
match csr_id with certificate of given partition and take decision if it shall
be signed or not.

This is "quite secure" apporach, a bit better than blidny trusting what CSR
to sign in KeDiFa. The bootstrap information, which is short living
(certificates are valid for 5 days), resides in SlapOS Master. The csr_id
is not directly known to SlapOS Master, and shall be consumed as fast as
possible by frontend cluster operator in order to sign CSR appearing in
KeDiFa caucase. The known possible attack vector requires that attacker knows
caucased HTTP listening port and can hijack HTTPS traffic to the csr_id-url
to get the human approve his own csr_id. The second is hoped to be overcomed
by publishing certificate of this endpoint via SlapOS Master.

Unfortunately caucase-updater prefix is directly used to find real CSR, as the
one generated is just a template for rerequest, thus csr_id would be different
from really used by caucase-updater.

Use KeDiFa to store keys, and transmit the url to the requester for master
and slave partitions.

Download keys on the slave partitions level.

Use caucase to fetch main caucase CA.

kedifa-caucase-url is published in order to have access to it.

Note: caucase is prepended with kedifa, as this is that one.

Use kedifa-csr tool to generate CSR and use caucase-updater macro.

Switch to KeDiFa with SSL Auth and updated goodies.

KeDiFa endpoint URLs are randomised.

Only one (first) user certificate is going to be automatically accepted. This
one shall be operated by the cluster owner, the requester of frontend master
partition.

Then he will be able to sign certificates for other users and also for
services - so each node in the cluster.

Special trick from https://security.stackexchange.com/questions/74345/provide-subjectaltname-to-openssl-directly-on-command-line
is used for one command generation of extensions in the certificate.
Note: We could upgrade to openssl 1.1.1 in order to have it really
simplified (see https://security.stackexchange.com/a/183973 )

Improve CSR readability by creating cluster-identification, which is master
partition title, and use it as Organization of the CSR.

Reserve slots for data exchange in KeDiFa.

Improvements:

 * support CSR as a file
   Allow to pass template_csr as a file, as it is useful for some cases.

 * use dumps where needed, as it is available

 * fix rerequest internal call

The certificates generated by caucase are not supported by Caddy (see https://www.erp5.com/project_section/vifib/forum/Problem-with-caddy-frontend-and-caucase-0.95-issued-certificate-UNinzubDv0)

/cc @rafael @alain.takoudjou

/reviewed-on nexedi/slapos!531

  As the instance is already allocated, we add a dummy template to keep data there until user do something.

  And to prevent to buildout keep failing.

/cc @rafael @jm @alain.takoudjou 

Here we go

/reviewed-on nexedi/slapos!529

  This helps on updating the 1.0 branch (release candidate) with the result of the latest tests.

share parts with instance-runner.cfg

/cc @rafael @Nicolas @alain.takoudjou 

/reviewed-on nexedi/slapos!527

As the test runs in erp5.util.testnode, which has some ports reserved,
and they collide with default ports of caddy-frontend services, select
ports for those services, and leave out default for monitor, as test expects.

use address 1 instead of address 0 and display a /etc/network/interfaces syntax.

Also display the info for resilient KVM.

Note that formatting is not perfect due to softwaretype recipe (which doesn't preserve spaces).

/cc @jm @rafael @alain.takoudjou

/reviewed-on nexedi/slapos!521

Unfortunately slave_title was put by mistake, it supposed to be slave_reference.

Use safe JSON serialisation/deserialisation, as otherwise unusual
slave_references can lead to issues and also character case is not kept.

Also care about case of log access user, which was undetected since
slave_reference in tests were always lowercase.

This reverts commit 1f91f19d.

Unfortunately due to way how profiles are mangled by jinja2, in some cases
the strings are becoming lowercased, so it just does not work.

It was not caught by tests, as no test has uppercase slave.

slave_title is dangerous, as it can contain any characters; it supposed
to be slave_reference.

Because there's no way anymore to pass the install prefix via an environment
variable, a new `make-install-extra` option (empty by default) is added so
that `make install` command line can be extended with `DESTDIR=...`.

/reviewed-on !524

Hardcoded partition names has been fixed after the shortening.

More changes are required to paths, as partition reference base has changed.

Also trafficserver starts in more cases, so move ERROR to OK state in some
promises.

At least trafficserver is known to not start on partitions on testnode like:

/srv/slapgrid/slappartX/srv/testnode/XXX/inst/test0-0/tmp/inst/TestSlaveGlobalDisableHttp2-1

so shorten the TestSlaveGlobalDisableHttp2 and others to simple T, which will result with path:

/srv/slapgrid/slappartX/srv/testnode/XXX/inst/test0-0/tmp/inst/T-1

Improved supervisord management is critical for maintenance of instances
between tests.

Since nexedi/slapos!392 we must format the json as done by `format-json`.

The tools for this are `format-json software/*/*.json` or `husky` ( nexedi/slapos!424 ).

/reviewed-on nexedi/slapos!522

fix json format

fix json format