I had a Theia instance where I installed buildout with `pip install -e --user /path/to/checkout` and later deleted that `/path/to/checkout` and this made it impossible to run instance buildout, failing with import error when importing zc.buildout  in buildout script.

This is because python load user site packages by default. I believe we don't want this, a broken user site package should not prevent slapos from running buildout.

These changes are about running buildout with `PYTHONNOUSERSITE` set, so that python ignores user site packages when running buildout.

references: 

 * https://docs.python.org/3/using/cmdline.html#envvar-PYTHONNOUSERSITE
 * https://docs.python.org/3/library/site.html#site.ENABLE_USER_SITE

See merge request nexedi/slapos.core!266

When PYTHONNOUSERSITE environemnt variable is set, python does not load user
site.
User site packages may contain a different version of buildout, which would
be selected when slapos runs buildout, causing unexpected behaviours.

Some buildout profiles are expecting this.

 - Use dict.items() to iterate on dict, instead of dict.keys() then dict[key]
 - don't use six.iterkeys(), this dict is small, so we don't need iterkeys()
   on python2 and can already write python3 version
 - don't % string when invoking logger, logging framework do it for us and do
   it only if message will be logged.

    For historical reason some aggregated packing lists could be merged (due a bug). This changes introduced backward compatibiltiy with those cases, rather them require a data migration.

    This is frequently updated on production and development instances.

Show exactly what type is used for a request, it makes debugging easier.

By default proxy checks if software type is None, then it's setting it to
RootSoftwareInstance, but if it comes as "" (empty string) it's kept as is.

So for consistency with proxy set default software type, so that proxy can
manage it correctly.

    This helps sync ZODB and mariadb (considering mariadb is newer). This is useful when restore a clone or synchronize after restore the mariadb catalog.

   The most appropriated way is reindex the site, however in case of slapos master (with large amount of data) it is too long (make this solution more appealing.

   If the Destroyed Hosting subscription is only present on Invalidated Open Orders, we just skip and converge the Hosting Subscription.

  This means the user deleted everything he had at once, too fast, before everything get processed.

See merge request nexedi/slapos.core!263

See merge request nexedi/slapos.core!262

See merge request nexedi/slapos.core!261

    ... else it leads to nowhere.

      He can access it or open a ticket.

Introduce new "Managed Resources" and assorted fixes accompanying nexedi/slapos!840 

Also respect meaning of verbose (more output in the console) and debug (drop in debugger on errors and keep things around for inspection)

See merge request nexedi/slapos.core!259

Really remove security_uid from roles_and_users table

Don't pollute console output, store this in log file

When we execute crontab commands, these commands might be python scripts, so
we don't want the PYTHONPATH set by `python setup.py test` to apply to these
scripts.

Respect semantics of "verbose" vs "debug", to make it possible to run tests in
debug mode but without verbose messages.

Also adjust "Starting", this message was not clear

No need to query the catalog multiple times for every partitions

In SlapOS software release tests we often have to start a process or store
temporary files in a directory for the lifetime of the test. Cleaning up the
resources is a bit error prone and source of code duplication.

This introduce a registry of resources that are automatically created and freed
by the test framework, to simplify tests.

See merge request nexedi/slapos.core!250

scan installed eggs and check if they have known vulnerabilities listed on
https://github.com/pyupio/safety-db

See merge request nexedi/slapos.core!255

cliff complete command only supports bash, but fish offers much more user friendly
completions.

cliff uses a pluggable system, but this where new shell completions can be registered,
but the plugin (CompleteFish) can not access the command manager to get more
details about options, since fish can also show help of suggestion, we want to
access the help of each option to show this during suggestions.
Also we want to make some more clever suggestions (for example slapos node start complete
with services identifiers).
For these reasons we need to register a command to replace the default complete command
and can not achieve this with simply a cliff.formatter.completion entry point.

This can be installed with:

    slapos complete > ~/.bash_completion.d/slapos
    slapos complete --shell fish > ~/.config/fish/completions/slapos.fish

github.com might be not available in many places.

In a scenario of an instance with child instances which does not properly
support destruction, a scenario like this happens:

 1. first report detect that it's destroying a root partition but child
partitions are still started and request destruction of these child instances
 2. child partitions are destroyed
 3. root partition is destroyed

For this case we need to run 3 times