The general idea of this patch is that now that we are using
zope.globalrequest, we no longer need to patch get_request, we can
simply call zope.globalrequest.setRequest with the request from the
test and restore the "real" request afterwards.

To achieve this, we reuse Testing.ZopeTestCase.connections.registry,
which already has the logic of cleaning up resources in the right place
and use a "Request" resource that calls setRequest(test_request) and
setRequest(real_request) when closed, so that:
 - test runs with an independant request
 - this test request is closed at the end
 - the real request is restored at the end

This also fixes a bug with self.publish when runnning
ERP5TypeLiveTestCase from portal_components of a running instance,
after a call to self.publish the current request was lost.

The testing for this revealed that ERP5TypeLiveTestCase.publish way
of dealing with zope.security interaction was not always correct: when
running a live test inside runUnitTest (like we do here in
testDynamicClassGeneration), there is no security interaction. This
was reviewed to use the high level API instead of changing directly the
internal storage.

This can be useful when making a report on movements and when we list
properties of the movements that depend on the side but are not
directly exposed on MovementHistoryListBrain. One use case was
`Movement_getSpecificReference`, which shows `source_reference` when
the brain is for the source and `destination_reference` otherwise.

With this new approach, instead of guessing we record the "is_source"
information at indexing time, when we know this for sure.

This also simplifies `MovementHistoryListBrain.date` and
`MovementHistoryListBrain.mirror_date` which no longer need to guess
the side and fix a problem that because this guessing was done using
`movement.getSourceUid()` - which cause security errors when users can
not access the source of the movement.

When editing a state permission mapping the roles were not sorted,
because WorkflowState_getPermissionMatrixContext uses a set. Sort
before setting the attribute, to prevent useless diffs in ZODB history
and business template.

default activate() behavior is to not queue the activity if another
activity with the same path/method_id already exist. We don't want this
here.

We don't even use the new updateLocalRolesOnSecurityGroups instead,
because roles are supposed to be applied by an interaction workflow.

to make it easier to find related documents

This compatibibility script was guessing the side, but this is exposed
in brain

See merge request !1768

The way addToDate was working with dates was not good, and creating
confusion when removing 1 month from the last days of a 31-day month, as
the previous day had less days than the current month:

date = DateTime(2023, 5, 31)
print date
print addToDate(date, month=-1)
> 2023/05/31 00:00:00 GMT+2
> 2023/05/01 00:00:00 GMT+2

This was even more confusing in March, with february having only 28
days:

date = DateTime(2023, 3, 31)
print date
print addToDate(date, month=-1)
> 2023/03/31 00:00:00 GMT+2
> 2023/03/03 00:00:00 GMT+2

The new behavior is to, when removing a month, if the new day of the new
month is more than the number of days in month to default to the last
day of the month. For exemple, removing one month from 31/05 becomes
30/04, and from there it will add/remove the days as necessary.

The real issue being that removing a month is ambiguous and can mean
a different thing for different people.

For reference, the reference implementation of timedelta in python
doesn't support adding months:

https://docs.python.org/3/library/datetime.html#datetime.timedelta

I hope my solution will make the more sense in ERP5's context.

For zope, a request to /erp5/person_module/person1 or
/erp5/organisation_module/person_module/person1 can both be resolved
and are (more or less) equivalent. This change brings support for
similar request paths for Open API Connectors.

( until we finalize the complete integration of SlapOS parameter
editor )

We set the module as a source to acquire roles. Also, we need to store request and response in HTTP Exchange.

This merge request is to fix ec4a3f9d

See merge request !1771

We set the module as a source to acquire roles. Also, we need to store request and response in HTTP Exchange.

This is a simple framework to implement services in ERP5 based on an
OpenAPI document.

A new type "Open API Type" (similar to "Base Type") is introduced,
this is responsible for the definition of operations and types.
The Open API document is set as text content of the Open API Type
and can be edited from the Open API Type.

For each service, a new portal type will be created. The portal type
use OpenAPIService as class and this is responsible for serving
requests. The process of serving requests is:
 - find the matching operation from the request method and request
   path
 - extracting request parameters and request body using the parameter
   definitions from the Open API Document
 - validate parameters and request body according to the schema from
   the Open API document
 - finding the method, this is done by using _getTypeBasedMethod with
   the operationId
 - calling the method and formatting the result or handling error.
   The default handling of errors is to reply with rfc7807 json
   responses, but it can be customized by defining an
   `handleException` type based method.

Typically, the services will be created in portal_web_services. From
there, there is also a view using a new SwaggerUI gadget to try out
the API.

What's not supported:
 - OpenAPI document in YAML format is only partially supported and
   have some limitations over JSON:
    - On python2 the order of operations is lost, the lookup of
      operations is not made in the order of the operations from the
      document. Also the operations are not in order in the SwaggerUI
      gadget.
    - The text editor does not provide rich editing of YAML
 - "partial" parameters in path elements ( /users/{user_id} is
   supported, but /documents/report.{format} is not )
 - XML (decoding of request bodies and parsing of responses) is not
   supported.

We use to show ID in case reference is missing, but since many years
this is no longer the case.

See merge request !1766

See merge request !1767

The fix from 7597c300 (accounting_l10n_fr: don't export
lines with 0 quantity in FEC, 2023-03-31) was not 100% correct,
lines with a quantity but with an asset price of 0 were still
exported.

We used a wrong condition that was causing lines with a debit
and asset credit (ie. different sides) to be output wrongly
with debit and credit.

in 5ae0508d (accounting_l10n_fr: Workaround encoding bugs
in Test Compta Demat, 2023-03-31) we replaced a few characters
that were known to be problematic, but while trying on some
other production data, we found new problematic characters.
This shown that this approach was too "optimistic" and that we
the safest way to have a file that is compatible with Test
Compta Demat is to use only ascii characters, this use
unicodedata to try to retain the original characters (so that
"Jérôme" becomes "Jerome") and also special case € to replace
it by "EUR"

    If two open orders are validated on the same transaction, archivePrevious script will get the an already archived as catalog will be outdated.

we pass the schema already, but for now this only supports basic
syntax highlighting

programmatically calling script.setParameterSignature('') was
setting _params to None and making it causing errors when calling
the script

This change modifies the FEC export to conform to the test software from
https://www.economie.gouv.fr/dgfip/outil-test-des-fichiers-des-ecritures-comptables-fec 
/ https://github.com/DGFiP/Test-Compta-Demat.

A new `test_compta_demat_compatibility` to enable compatibility mode was introduced.
This will be True when rendering the report from the dialog, i.e., using `AccountingTransactionModule_viewFrenchAccountingTransactionFile` and False when
using `AccountingTransactionModule_viewFrenchAccountingTransactionFileActive` directly
so the behavior of reusing FEC export for accounting interfaces should be the same as before.


See merge request nexedi/erp5!1764

> IX. – Le fichier des écritures comptables est nommé selon la
> nomenclature suivante :
>
> SirenFECAAAAMMJJ, où " Siren " est le Siren du contribuable mentionné
> à l'article L. 47 A et AAAAMMJJ la date de clôture de l'exercice
> comptable.

according to https://bofip.impots.gouv.fr/bofip/9028-PGP.html/identifiant%3DBOI-CF-IOR-60-40-20-20170607#III._Donnees_devant_figurer_12

> Dans le cas des écritures pour lesquelles il n’existe pas de référence
> de pièce (par exemple, dans le cas des écritures d’à nouveau), ce
> champ doit néanmoins être rempli. Dans ce cas précis, la référence
> sera alors remplacée par une valeur conventionnelle définie par
> l’entreprise. Celle-ci sera précisée dans le descriptif remis au
> vérificateur en même temps que le fichier des écritures comptables
> (cf. VI § 390).

La valeur conventionnelle dans cet export est de donner à PieceRef la
même valeur que EcritureNum pour les écritures sans référence de pièce.

"Test Compta Demat" complains about these
https://github.com/DGFiP/Test-Compta-Demat/blob/1563a42a260e754a7543b2ee178d4300d1582db5/src/testeur/trt_txt.pl#L1023-L1033

This removes some characters causing 'Erreur Wide character in
subroutine entry' bugs in Test Compta Demat from the XML output

The test software from
https://www.economie.gouv.fr/dgfip/outil-test-des-fichiers-des-ecritures-comptables-fec
only validates if the XSD is referenced with xsi:noNamespaceSchemaLocation

See merge request nexedi/erp5!1763

XLTE(*) follows the same top-level packaging structure like
wendelin.core does.

Registration is done via new `register_xpkg` function instead of just
loop so that pkgname argument is properly captured in created lambda and
does not change after going to next iteration.

(*) https://lab.nexedi.com/kirr/xlte

/reviewed-by @jerome
/reviewed-on nexedi/erp5!1762

In 5796a17a (core_test: Add test to make sure that wendelin.core
basically works) I added wendelin_transform to pylint so that the
checker could recognize wendelin.core's special wendelin top-level
module/package and handle ok

	from wendelin.bigarray.array_zodb import ZBigArray

this works, but e.g. for plain

	import wendelin			or
	from wendelin import bigarray

pylint currenly fails with

	AssertionError: explicit relative import, but no context_file?

To better see what is going on let's conside the following test program

	"""ZZZ"""

	from wendelin.bigarray.array_zodb import ZBigArray
	from wendelin import bigarray

	def main():
	    """main"""
	    _ = ZBigArray

and run pylint on it. Here is what it gives:

          wendelin_transform Module(wendelin)
          -> Module()

    visit_from From()
      basename: wendelin.bigarray.array_zodb
      names:    [('ZBigArray', None)]
      modnode:  Module(test)
        get_imported_module From() wendelin.bigarray.array_zodb
      importedmodnode: Module(wendelin.bigarray.array_zodb)
    ('_add_imported_module', <From() l.3 [test] at 0x7ff214b4cb90>, 'wendelin.bigarray.array_zodb.ZBigArray')
          wendelin_transform Module(wendelin)
          -> Module()

    visit_from From()
      basename: wendelin
      names:    [('bigarray', None)]
      modnode:  Module(test)
        get_imported_module From() wendelin
          wendelin_transform Module(wendelin)
          -> Module()
      importedmodnode: Module()
    ************* Module test
    W:  4, 0: Relative import 'wendelin', should be '' (relative-import)
    ('_add_imported_module', <From() l.4 [test] at 0x7ff214b4cf50>, '.bigarray')
    Traceback (most recent call last):
      File "./x.py", line 22, in <module>
        Run(['test.py', '--reports=n'], exit=False)
      File "/home/kirr/src/tools/py/pylint/pylint/lint.py", line 1332, in __init__
        linter.check(args)
      File "/home/kirr/src/tools/py/pylint/pylint/lint.py", line 747, in check
        self._do_check(files_or_modules)
      File "/home/kirr/src/tools/py/pylint/pylint/lint.py", line 869, in _do_check
        self.check_astroid_module(ast_node, walker, rawcheckers, tokencheckers)
      File "/home/kirr/src/tools/py/pylint/pylint/lint.py", line 946, in check_astroid_module
        walker.walk(ast_node)
      File "/home/kirr/src/tools/py/pylint/pylint/utils.py", line 874, in walk
        self.walk(child)
      File "/home/kirr/src/tools/py/pylint/pylint/utils.py", line 871, in walk
        cb(astroid)
      File "/home/kirr/src/tools/py/pylint/pylint/checkers/imports.py", line 288, in visit_from
        self._add_imported_module(node, '%s.%s' % (importedmodnode.name, name))
      File "/home/kirr/src/tools/py/pylint/pylint/checkers/imports.py", line 328, in _add_imported_module
        importedmodname = get_module_part(importedmodname)
      File "/home/kirr/src/tools/py/astroid/astroid/modutils.py", line 359, in get_module_part
        'explicit relative import, but no context_file?'
    AssertionError: explicit relative import, but no context_file?

we see that the line

	from wendelin.bigarray.array_zodb import ZBigArray

corresponds to

	('_add_imported_module', <From() l.3 [test] at 0x7ff214b4cb90>, 'wendelin.bigarray.array_zodb.ZBigArray')

and the line

	from wendelin import bigarray

corresponds to

	('_add_imported_module', <From() l.4 [test] at 0x7ff214b4cf50>, '.bigarray')

notice that in the latter case there is no 'wendelin' prefix and the
import goes as just '.bigarray' instead of 'wendelin.bigarray' which
leads to further crash in get_module_part.

-> Fix it by initializing wendelin module yielded by wendelin_transform
with .name set. Not sure why it is working longer imports without that.

Without the fix added test breaks as

    .../instance/slappart5/bin$ ./runUnitTest -v -v -v  testDynamicClassGeneration.TestZodbModuleComponent
    ...
    Running Unit tests of <class 'testDynamicClassGeneration.TestZodbModuleComponent'>
    ok
    testImportVersionedComponentOnly (testDynamicClassGeneration.TestZodbModuleComponent) ... ok
    testInvalidId (testDynamicClassGeneration.TestZodbModuleComponent) ... ok
    testInvalidSourceCode (testDynamicClassGeneration.TestZodbModuleComponent) ... ok
    testModuleSecurityInfo (testDynamicClassGeneration.TestZodbModuleComponent) ... ok
    testPylint (testDynamicClassGeneration.TestZodbModuleComponent) ... FAIL
    testPylintAstroidModuleGeneratedOnce (testDynamicClassGeneration.TestZodbModuleComponent) ... ok
    testPylintNamedtupleUnicodeLiteralsRegression (testDynamicClassGeneration.TestZodbModuleComponent) ... ok
    testReferenceWithReservedKeywords (testDynamicClassGeneration.TestZodbModuleComponent) ... ok
    testValidateInvalidateDelete (testDynamicClassGeneration.TestZodbModuleComponent) ... ok
    testVersionPriority (testDynamicClassGeneration.TestZodbModuleComponent) ... ok
    testVersionWithReservedKeywords (testDynamicClassGeneration.TestZodbModuleComponent) ... ok
    testWorkflowErrorMessage (testDynamicClassGeneration.TestZodbModuleComponent)
    Check that validation error messages are stored in workflow ... ok

    ======================================================================
    FAIL: testPylint (testDynamicClassGeneration.TestZodbModuleComponent)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File ".../parts/erp5/Products/ERP5Type/tests/testDynamicClassGeneration.py", line 2347, in testPylint
        component.checkSourceCode()
      File ".../parts/erp5/product/ERP5Type/mixin/component.py", line 329, in checkSourceCode
        return checkPythonSourceCode(self.getTextContent(), self.getPortalType())
      File ".../parts/erp5/product/ERP5Type/Utils.py", line 540, in checkPythonSourceCode
        Run(args, reporter=TextReporter(output_file), exit=False)
      File ".../develop-eggs/pylint-1.4.4+slapospatched002-py2.7.egg/pylint/lint.py", line 1332, in __init__
        linter.check(args)
      File ".../develop-eggs/pylint-1.4.4+slapospatched002-py2.7.egg/pylint/lint.py", line 747, in check
        self._do_check(files_or_modules)
      File ".../develop-eggs/pylint-1.4.4+slapospatched002-py2.7.egg/pylint/lint.py", line 869, in _do_check
        self.check_astroid_module(ast_node, walker, rawcheckers, tokencheckers)
      File ".../develop-eggs/pylint-1.4.4+slapospatched002-py2.7.egg/pylint/lint.py", line 946, in check_astroid_module
        walker.walk(ast_node)
      File ".../develop-eggs/pylint-1.4.4+slapospatched002-py2.7.egg/pylint/utils.py", line 874, in walk
        self.walk(child)
      File ".../develop-eggs/pylint-1.4.4+slapospatched002-py2.7.egg/pylint/utils.py", line 871, in walk
        cb(astroid)
      File ".../develop-eggs/pylint-1.4.4+slapospatched002-py2.7.egg/pylint/checkers/imports.py", line 253, in visit_import
        self._add_imported_module(node, importedmodnode.name)
      File ".../develop-eggs/pylint-1.4.4+slapospatched002-py2.7.egg/pylint/checkers/imports.py", line 319, in _add_imported_module
        importedmodname = get_module_part(importedmodname)
      File ".../develop-eggs/astroid-1.3.8+slapospatched001-py2.7.egg/astroid/modutils.py", line 359, in get_module_part
        'explicit relative import, but no context_file?'
    AssertionError: explicit relative import, but no context_file?

    ----------------------------------------------------------------------
    Ran 13 tests in 137.609s

    FAILED (failures=1)

/helped-and-reviewed-by @jerome
/reviewed-on nexedi/erp5!1762

See merge request nexedi/erp5!1759