See merge request !360

See merge request nexedi/slapos.core!356

   This introduce API for request support and add events on Support Requests

   Used to create support requests to send users notifications.

   This creates a Support Request

fixup for a8917a3c

Check that promises run with the instance's python are logged properly.

Promises processed using the instance's python in a subprocess were
logged to the console but not to the logfile passed by `--logfile`.

To fix this the output of the promise is now captured and forwarded
to slapgrid's logger in real-time using SlapPopen.

fixup for a8917a3c

See merge request nexedi/slapos.core!365

The option use_unique_local_address was added in
de11308a but was already broken because
2 different names were used for the option:
 * use_unique_local_address_block
 * use_unique_local_address

There was a tentative fix in c09214cb
but the option name in slapos.cfg.example was never fixed.

The c647aab4 dropped _storeLastData, so
follow-up changes here.

Wrong object used resulted with non working SupportRequest_recheckMonitoring
on Compute Node.

   It wasn't working properly, let's reintroduce once it better tested.

Before slapos commit a184cca8b (Try to reuse existing file to avoid
excessive IO on update + other minor optimisations, 2019-06-17), which
started to be used in profiles in 65acdca0d (Release slapos.cookbook
(1.0.118), 2019-08-13), the promise plugins did not contain

    import sys
    sys.path[0:0] = (..., )

but there was an extra import statement:

    import sys
    import json
    sys.path[0:0] = (..., )

slapos.core >= 1.7 was not able to process such promises, because the
code introspecting these promises assumed that the second node of the
AST was the assignment, which is incorrect in the old promises because
it was another import.

To accomodate both cases, use the value from the first assignment.

We also change to use ast.literal_eval for safety.

This method returns the class of computer used for the test, then
it's easier for test subclasses to use a computer with different
classes for software and instances.

Use this in TestSlapgridPluginPromiseWithInstancePython, using
subclasses instead of monkey patching should make extending this
test a bit easier

See merge request nexedi/slapos.core!359

   This API replaces the usage of SlapTool code for handling data stored in memcached. It introduces an New API for it at the context of the objects to handle volatile data used on SlapOS context.

   Replace Base_getNewsDict by getAccessStatus

   Drop direct access to memcache, use Cache Plugin instead to load and
set information.

    In case the instance isn't linked to the tree properly (due Garbage Collect unlinked it), stop the instance directly anyway.

Once per day is not real time monitoring, so drop it to each 5 minutes.

On python2, warnings.warn expects the message to be passed as a str
and it will silently output nothing when the message is an unicode
that can not be converted to str using ascii encoding (cf.
https://bugs.python.org/issue34752 )

Since october update of pyupio/safety-db there is a new vulnerability
for pytest-runner, which is described as:

    "pytest-runner": [
        {
            "advisory": "Pytest-runner depends on deprecated features of setuptools and relies on features that break security mechanisms in pip. For example \u2018setup_requires\u2019 and \u2018tests_require\u2019 bypass pip --require-hashes. See also pypa/setuptools#1684.\r\nIt is recommended that you:\r\n- Remove 'pytest-runner' from your setup_requires, preferably removing the setup_requires option.\r\n- Remove 'pytest' and any other testing requirements from tests_require, preferably removing the tests_requires option.\r\n- Select a tool to bootstrap and then run tests such as tox.",
            "cve": "PVE-2021-43313",
            "id": "pyup.io-43313",
            "specs": [
                ">0"
            ],
            "v": ">0"
        }
    ],

notice the quotes:

    \u2018setup_requires\u2019 and \u2018tests_require\u2019

this was sent to warnings.warn as unicode and because all our softwares
have this vulnerability (that BTW do not impact us because we run
buildout with a patch for setup_requires), we no longer saw any
warning on python2 software release tests.

pkg_resources.Environment only scans packages for the current python
version, but the checked software might be using a different python
version than the python version used to check software (the slapos
python). The checked software might also include packages for
multiples python versions.

Compute the list used python packages from the eggs directory and use
an Environment for each python version, this way we check all python
versions and also when slapos python and software python are different.

 - only consider shared parts from checked software
 - support new `.buildout-shared.json` signature files
 - unit test coverage

See merge request nexedi/slapos.core!361

See merge request nexedi/slapos.core!358

check_software was checking all shared parts, which was an easy
implementation but turned out to be problematic, both when running on
test nodes because if one software installs broken shared parts all
softwares being tested afterwards were failing and also during
development because we needed to prune broken shared parts when
fixing problems reported by check_software.

This inspects buildout's .installed.cfg to guess which shared parts
are used.