- test extended

If automatic GC happens just after the testnode repository is updated,
the repository in the SR may lose objects and fail to update.

See commit 9634a7d8.

Pylint warnings are now causing test failures.

/reviewed-on nexedi/erp5!987

revision(s) a Test Result has already been completed.
Thus we need to properly handle this use case otherwise simply failing makes "client
side" - i.e. testnode keep trying forever in its _retryRPC calls which
indefinitely blocks further tests executions.

A new test suite to perform static check on business templates using  `CodingStyleTestCase`.

This runs a coding style test for each business template, after installing the business template and its dependencies listed in business template metadata.

---

In order to start with a test suite with no failure, business templates that does not pass this test today (because they don't install or because pylint issues still needs to be reviewed) are skipped with an ad-hoc  `bt/skip_coding_style_test` in the business template. Note that this file is not created by business template system as it is a temporary measure, business templates must pass pylint and other checks from `CodingStyleTestCase`.

---

I took the approach of creating one independent test per business template, instead of the approach of creating a site with all business templates like we did in testNamingConventions or testHTML. This appears as another "Coding Style" test, like Performance or Scalability tests.

The idea behind this was that if we extend our business template checks ( for example when we translate the [ERP5 Module Creation Guidelines](https://www.erp5.com/documentation/developer/guideline/module) to some scripts we can run on a business template ), we can check that each business template contain what it should contain and not just that the global result is OK.



/reviewed-on !629

PortalTransforms: safe_html: Changes in b255c894 were not actually applied so merge FS module and portal_transforms/safe_html.

These titles duplicate listbox configuration, preventing any change from
reaching users (...who customised their listbox).
Also fixes cases where column title was stored in selection after
translation, which breaks some reports.

Avoid double-calls when one is enough.
Avoid unused locals.
Avoid repeating conditions.
Inline single-use expressions.

Bootstrap is actually the automatic migration of tools. It happened to me
several times that I mistakenly start an old instance on a too new version of
products and this instance is migrated automatically.

By leaving a note in transaction it's easier to find out which
transaction performed the automatic migration to revert to the state
before using technique such as
https://blog.gocept.com/2019/06/14/undo-transactions-by-truncating-zodb-data-fs/
for FileStorage.

with this change, a command such as;

    python -m ZODB.scripts.fstail -n 20 ./instance/slappart6/srv/zodb/root.fs

show something like:

    2019-11-12 02:46:42.106313: hash=44fecb9a63411fbf305cea82f4c699c478000d61
    user=' None' description='/timer_service/process_timer\nSite migrated' length=462938 offset=881877846 (+70)

for the transaction which did the migration, so it's easy to repair.

/reviewed-on nexedi/erp5!975

* erp5_dms: ERP5OOo/ConversionServer-related Transforms.
* erp5_web: Web Page-related Transform.
* erp5_base: Everything else.

Also, ZODB Components Transforms modules were 'Document Component', whereas
it should be 'Module Component' (introduced recently) considering that they
have no Portal Type.

We'll gradually enable the business templates as we fix them.

See commit a93ea5f1.

Test cancelling a dialog displayed twice.
Test submitting a dialog displayed twice.

Keep compatibility with officejs apps

Compatibility

ZODB Components: PortalTransforms: No need to reload() 'erp5.' dynamic modules as this done through reset.

Calling manage_reloadAllTransforms was raising KeyError.

ZODB Components: image_to_{bmp,pcx} have already been migrated but the filesystem modules still existed.

You may have to either restart your instance or call portal_transforms/manage_reloadAllTransforms
for this change to take effect.

This is a mirror of the late import of getCurrentNode within
product/ERP5Type/Core/Folder.py
Both are required depending on which module is imported first.

This reverts commit 0af3e8f6.

This is not required anymore since ERP5Type.PropertySheet is a dynamic module
(commit 9ddcbfc1).

from a41f71c3 :
> Remove the TALES for lines, columns and tab on the matrix box because
> the default behavior is to call this updateCellRange script

that's not true, we have to list the asCellRange script explicitly.

ZODB Components: ModuleSecurityInfo() should also apply to the alias module, not only to the versioned one.

This fixes an Unauthorized error when GitLoginError is imported from
erp5.component.module.Git defining ModuleSecurityInfo() for __name__
(erp5.component.module.erp5_version.Git).

Note that when migrating the content of a Product, ModuleSecurityInfo() first
parameter must be __name__.

Traceback (innermost last):
  [...]
  Module script, line 1, in BusinessTemplate_handleException
   - <PythonScript at /erp5/BusinessTemplate_handleException used for /erp5/portal_templates/832>
   - Line 1
    from erp5.component.module.Git import GitLoginError
  Module Products.ERP5Type.patches.Restricted, line 305, in guarded_import
    return orig_guarded_import(mname, globals, locals, fromlist, level)
  Module AccessControl.ZopeGuards, line 305, in guarded_import
    raise Unauthorized("import of '%s' is unauthorized" % mname)
Unauthorized: import of 'erp5.component.module.Git' is unauthorized

ZODB Components: Alias module was not added to sys.modules if the module it points to had already been loaded before.

- revert parallel gadget loading
- minor refactoring on appcache storage creation

- new skin folder with project quick view form, scripts and gadgets
- front page redirects to custom project module

with namedtuples and unicode literals enabled, our patch break with:

    Module py2stdlib, line 266, in infer_named_tuple
      ''' % {'name': name, 'fields': attributes})
    Module Products.ERP5Type.patches.pylint, line 74, in string_build
      encoding = _guess_encoding(data)
    Module astroid.builder, line 65, in _guess_encoding
      if string.startswith('\xef\xbb\xbf'):
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position 0: ordinal not in range(128)

because pylint/astroid uses string_build to evaluate the named tuple and the source code enabled 
unicode_literals future, string_build is called with an unicode object.

    -> for inferred in node.func.infer():
      astroid-1.3.8-py2.7.egg/astroid/bases.py(391)infer()
    -> return self._explicit_inference(self, context, **kwargs)
      astroid-1.3.8-py2.7.egg/astroid/brain/py2stdlib.py(266)infer_named_tuple()
    -> ''' % {'name': name, 'fields': attributes})
    > erp5/product/ERP5Type/patches/pylint.py(77)string_build()
    -> data = data.encode('utf-8')
    (Pdb) up
    > astroid-1.3.8-py2.7.egg/astroid/brain/py2stdlib.py(266)infer_named_tuple()
    -> ''' % {'name': name, 'fields': attributes})
    (Pdb) l
    249     def infer_named_tuple(node, context=None):
    250         """Specific inference function for namedtuple CallFunc node"""
    251         class_node, name, attributes = infer_func_form(node, nodes.Tuple._proxied,
    252                                                        context=context)
    253         fake = AstroidBuilder(MANAGER).string_build('''
    254     class %(name)s(tuple):
    255         _fields = %(fields)r
    256         def _asdict(self):
    257             return self.__dict__
    258         @classmethod
    259         def _make(cls, iterable, new=tuple.__new__, len=len):
    260             return new(cls, iterable)
    261         def _replace(_self, **kwds):
    262             result = _self._make(map(kwds.pop, %(fields)r, _self))
    263             if kwds:
    264                 raise ValueError('Got unexpected field names: %%r' %% list(kwds))
    265             return result
    266  ->     ''' % {'name': name, 'fields': attributes})
    (Pdb) pp name
    u'NamedTuple'
    (Pdb) pp attributes
    [u'foo', u'bar']
    (Pdb) pp [ (arg, arg.value) for arg in node.args ]
    [(<Const(unicode) l.4 [checkPythonSourceCodelXOzr3] at 0x7f9f1caee250>,
      u'NamedTuple'),
     (<Const(unicode) l.4 [checkPythonSourceCodelXOzr3] at 0x7f9f1caeebd0>,
      u'foo bar')]
    
    

/reviewed-on nexedi/erp5!978

To avoid duplicating _processSearchValue.

Scriptable keys are already handled by other branches below.
If one wants to provide a string or dict value to a scriptable key, they
need to wrap it into another type to disable catalog processing of these
values.