1. 13 Jun, 2022 2 commits
  2. 09 Jun, 2022 6 commits
    • Jérome Perrin's avatar
      Enable coding style on erp5_payroll · 34d50a78
      Jérome Perrin authored
      See merge request nexedi/erp5!1637
      34d50a78
    • Jérome Perrin's avatar
      core: repair ERP5Site_reindexLatestIndexedObjects · e83e7960
      Jérome Perrin authored
      This script stopped working since Catalog changed to be ERP5 document,
      because it was calling the unindex method of the catalog (which
      itself is indexable like any other ERP5 document). Update to use the
      uncatalogObject which is the method to unindex a document by uid.
      
      Even though it was somehow working before, it was not really correct
      in selecting objects, because it was using - operator on TIMESTAMP
      column, which is not computing a difference in seconds as this script
      was expecting. See for example https://stackoverflow.com/a/24504132/7607763
      or the example below for an explanation. Instead, use TIMESTAMPADD
      to compute the start timestamp only once and use >= operator, which
      works as expected.
      
      This query was also sorting by indexation_timestamp, which does not
      use an index. Remove the sort because it's not really needed.
      
      Excluding reserved path was also not needed, we no longer use these
      since 69aefdff (ZSQLCatalog: Drop support for "reserved" path.,
      2017-09-18)
      
      ---
      
      Another reproduction of the timestamp arithmetic problem
      
      select
         TIMESTAMP('2021-01-02 00:00:00') - TIMESTAMP('2021-01-01 00:00:00') a,
         20210102000000 - 20210101000000 aa,
         TIMESTAMP('2021-06-01 00:00:00') - TIMESTAMP('2021-05-31 00:00:00') b,
         20210601000000 - 20210531000000 bb,
         TIMESTAMPDIFF(second, TIMESTAMP('2021-05-31 00:00:00'), TIMESTAMP('2021-06-01 00:00:00')) c
      
      | a | aa | b | bb | c |
      | ------ | ------ | ------ | ------ | ------ |
      | 1000000 | 1000000 | 70000000 | 70000000 | 86400 |
      e83e7960
    • Jérome Perrin's avatar
      9a518828
    • Jérome Perrin's avatar
      tests/SecurityTestCase: sort groups in assertion failure message · f1e4e1c3
      Jérome Perrin authored
      This is a bit easier to read
      f1e4e1c3
    • Jérome Perrin's avatar
      74157942
    • Jérome Perrin's avatar
      Zelenium: catch potential errors when making (HTML) link for failed test · bb7edb27
      Jérome Perrin authored
      When a test fail, we make a data-url link with the HTML of the current
      page, so that we can easily investigate test failures n test nodes.
      We should not let errors that might happen here propagate, otherwise
      the test result is not created and the test runner does not detect
      that the test is finished.
      
      One case that caused such errors was failed assertion just after
      using goBack command without waiting
      bb7edb27
  3. 08 Jun, 2022 2 commits
  4. 06 Jun, 2022 6 commits
  5. 01 Jun, 2022 1 commit
  6. 27 May, 2022 1 commit
    • Jérome Perrin's avatar
      AlarmTool: handle automatic solve with alarms owned by system user · 58f6b8dc
      Jérome Perrin authored
      Business templates are installed by system user, which is a special
      user not returned by getWrappedOwner. Because of this, the "fixing
      problems or activating a disabled alarm is not allowed" error was
      raised when checking if the owner of the alarm has manage portal
      permission on the alarm.
      
      This switches the implementation to explicit creation of the user
      when user id is the system user, so that we have a user with the
      permission to solve the alarm.
      58f6b8dc
  7. 26 May, 2022 5 commits
  8. 24 May, 2022 6 commits
    • Valentin Benozillo's avatar
      erp5_ui_test: Drop the plural · d17614c7
      Valentin Benozillo authored
      d17614c7
    • Valentin Benozillo's avatar
      ui_test: Add range browser error test · 3998056d
      Valentin Benozillo authored
      When start and end are setup on float/integer field, the max and mic attributes are also setup in the input html element.
      So if the user enter a number outside this range, the browser should display an error.
      3998056d
    • Valentin Benozillo's avatar
      erp5_ui_test: Add external validator for float field range check · 789da4e7
      Valentin Benozillo authored
      using start and end filed on float, trigger web-browser error. But UI test are looking for multiple fields check by erp5.
      2 validators script are added because no argument can be passed to external validator, so :
      TALES: python: context.Base_ValidateFloatRange(value, mix=1, max=2)
      doesn't work
      789da4e7
    • Jérome Perrin's avatar
      *: keep using func_code and not yet __code__ with scripts · eaae74a0
      Jérome Perrin authored
      On Zope2, python scripts do not have __code__, they only have
      func_code (and same for __defauls__/func_defaults).
      We tried to backport the support of __code__ from Zope4 as a Zope2
      patch - it was SlapOS patch 4fa33dfc6 (erp5: py3: `func_{code,defaults}`
      was replaced in Python3 by `__{code,defaults}__`., 2022-04-25),
      but this patch was incomplete. We tried to backport more, but then
      realized that we don't need to use __code__ on ERP5 master yet,
      because ERP5 master branch is still supporting Zope2 only.
      
      This patch revert a small part of a17bb910 (py2/py3: Make Products
      code compatible with both python2 and python3., 2022-04-13), the part
      where we use f.__code__ where f might be a python script. For now,
      we'll apply this patch only on the Zope4 branch.
      
      A few places where f.func_code was used and f was a for sure not a
      python script but a simple class method or function are kept here, as
      __code__ support is missing only on in ZODB scripts.
      eaae74a0
    • Jérome Perrin's avatar
      tests: infrastructure to rebuild and export all business templates · 83e69b6b
      Jérome Perrin authored
      This is not really a test, but it reuses runUnitTest/runTestSuite
      commands, because they are good tools to quickly create ERP5
      environment and installing business templates.
      
      To re-build and re-export all* business templates, use this command:
      
          ./bin/runTestSuite --test_suite=ReExportERP5BusinessTemplateTestSuite
      
      --node_quantity argument can also be used to process multiple
      business templates in parallel.
      
      * note that this does not actually handle all business templates, but
      only the ones for which coding style test is enabled, because most
      business templates for which it is not enabled can not be installed.
      
      This typically produces large diffs that should apply the same
      change to many files and ideally, nothing else. We also developed a
      simple tool which summarize the diff by detecting the same chunk
      present in multiple files, it can be found at
      https://lab.nexedi.com/nexedi/erp5/snippets/1171 and also below.
      
      ---
      
      from __future__ import print_function
      """report similar hunks in a patch.
      """
      __version__ = '0.1'
      
      import argparse
      import collections
      import codecs
      import unidiff # unidiff==0.7.3
      import hashlib
      
      parser = argparse.ArgumentParser()
      parser.add_argument('patch_file', type=argparse.FileType('r'), default='-', nargs='?')
      parser.add_argument('-v', '--verbose', action='count', default=0)
      args = parser.parse_args()
      
      patchset = unidiff.PatchSet(codecs.getreader('utf-8')(args.patch_file))
      
      chunks_by_filenames = collections.defaultdict(set)
      
      for patch in patchset:
        for chunk in patch:
          chunk_text = u''.join([unicode(l) for l in chunk])
          chunks_by_filenames[chunk_text].add(patch.path)
      
      for chunk_text, filenames in chunks_by_filenames.items():
        chunk_hash = hashlib.md5(chunk_text.encode('utf-8')).hexdigest()
        print("Chunk %s is present in %s files" % (chunk_hash, len(filenames)))
        if args.verbose:
          print()
          print("\n".join("  " + f for f in sorted(filenames)))
          print()
        if args.verbose > 1:
          print()
          print(chunk_text)
          print()
      83e69b6b
    • Jérome Perrin's avatar
      accounting: always compare rounded values when indexing balance transactions · b075c99b
      Jérome Perrin authored
      To prevent rounding errors, we always compare rounded values to the
      precision of the accounting currency. There was a place here where we
      were using -= without rounding, which caused to detect a difference
      between new and current stock and insert a line for 0 in an existing
      instance for which some accounting lines were created with too precise
      values - but not in a way that was detected by the assertions in
      AccountingPeriod_createBalanceTransaction.
      
      Rounding here as well solved the problem with that data.
      b075c99b
  9. 23 May, 2022 2 commits
  10. 21 May, 2022 1 commit
    • Jérome Perrin's avatar
      ERP5Type/patches: prepare for removal of Products.DCWorkflowGraph · 88321109
      Jérome Perrin authored
      Supports the case where Products.DCWorkflowGraph is not present.
      Even though we are removing Products.DCWorkflowGraph from the
      software release, we don't remove this monkey patch yet, because
      this monkey patch also fixed a severe security issue. We keep the
      patch for the cases where a recent ERP5 runs on an old SlapOS where
      the product is still there.
      
      This change just moves the existing code in a try/except ImportError
      block
      88321109
  11. 19 May, 2022 1 commit
  12. 18 May, 2022 4 commits
    • Jérome Perrin's avatar
      85317472
    • Levin Zimmermann's avatar
      restricted: Allow patched pandas.read_* functions · 4360dbc6
      Levin Zimmermann authored
      Rationale:
      
      Converting * to data frame / numpy array efficiently is required in all
      wendelin projects, without this functionality wendelin is useless.
      Currently all projects allow this functionality in an insecure way.
      This commit aims to improve the situation by supporting a secure way of
      this functionality.
      
      (See nexedi/wendelin!99 (comment 158474))
      
      Because pandas (in restricted Python) can also be useful in 'pure' ERP5
      (without Wendelin) the functionality is added to ERP5 source code.
      
      ---
      
      Security:
      
      Security is guaranteed by patching selected read_* functions and
      allowing the patched versions. The patch prohibits anything but
      string input which directly contains the data (e.g. no urls, file
      paths). New unit tests ensure the restrictions of the patches
      are actually effective.
      
      ---
      
      Notes on implementation decisions:
      
      Instead of offering new ERP5 extension methods (e.g. Base_readJson)
      this commit adds patched pandas read functions in restricted Python.
      In this way the change of the known API is as minimal as possible.
      
      Instead of globally monkey-patching pandas read_* functions, only the
      functions inside restricted python are patched.
      In this way the fully-functional, original functions are still available
      in Zope products or ERP5 extension code.
      
      Minor changes in the way how pandas is allowed in restricted python
      have been applied. Please consult the following discussions in the Merge
      request for details:
      
      nexedi/erp5!1615 (comment 159203)
      nexedi/erp5!1615 (comment 159341)
      4360dbc6
    • Levin Zimmermann's avatar
      70b92437
    • Levin Zimmermann's avatar
      ERP5Site: Remove compatibility with old data model · b1bdb286
      Levin Zimmermann authored
      ...for  getPortalDataConfigurationTypeList.
      
      See !1630 (comment 159889).
      b1bdb286
  13. 17 May, 2022 3 commits