1. 09 Jun, 2022 5 commits
    • Jérome Perrin's avatar
      core: repair ERP5Site_reindexLatestIndexedObjects · e83e7960
      Jérome Perrin authored
      This script stopped working since Catalog changed to be ERP5 document,
      because it was calling the unindex method of the catalog (which
      itself is indexable like any other ERP5 document). Update to use the
      uncatalogObject which is the method to unindex a document by uid.
      
      Even though it was somehow working before, it was not really correct
      in selecting objects, because it was using - operator on TIMESTAMP
      column, which is not computing a difference in seconds as this script
      was expecting. See for example https://stackoverflow.com/a/24504132/7607763
      or the example below for an explanation. Instead, use TIMESTAMPADD
      to compute the start timestamp only once and use >= operator, which
      works as expected.
      
      This query was also sorting by indexation_timestamp, which does not
      use an index. Remove the sort because it's not really needed.
      
      Excluding reserved path was also not needed, we no longer use these
      since 69aefdff (ZSQLCatalog: Drop support for "reserved" path.,
      2017-09-18)
      
      ---
      
      Another reproduction of the timestamp arithmetic problem
      
      select
         TIMESTAMP('2021-01-02 00:00:00') - TIMESTAMP('2021-01-01 00:00:00') a,
         20210102000000 - 20210101000000 aa,
         TIMESTAMP('2021-06-01 00:00:00') - TIMESTAMP('2021-05-31 00:00:00') b,
         20210601000000 - 20210531000000 bb,
         TIMESTAMPDIFF(second, TIMESTAMP('2021-05-31 00:00:00'), TIMESTAMP('2021-06-01 00:00:00')) c
      
      | a | aa | b | bb | c |
      | ------ | ------ | ------ | ------ | ------ |
      | 1000000 | 1000000 | 70000000 | 70000000 | 86400 |
      e83e7960
    • Jérome Perrin's avatar
      9a518828
    • Jérome Perrin's avatar
      tests/SecurityTestCase: sort groups in assertion failure message · f1e4e1c3
      Jérome Perrin authored
      This is a bit easier to read
      f1e4e1c3
    • Jérome Perrin's avatar
      74157942
    • Jérome Perrin's avatar
      Zelenium: catch potential errors when making (HTML) link for failed test · bb7edb27
      Jérome Perrin authored
      When a test fail, we make a data-url link with the HTML of the current
      page, so that we can easily investigate test failures n test nodes.
      We should not let errors that might happen here propagate, otherwise
      the test result is not created and the test runner does not detect
      that the test is finished.
      
      One case that caused such errors was failed assertion just after
      using goBack command without waiting
      bb7edb27
  2. 08 Jun, 2022 2 commits
  3. 01 Jun, 2022 1 commit
  4. 27 May, 2022 1 commit
    • Jérome Perrin's avatar
      AlarmTool: handle automatic solve with alarms owned by system user · 58f6b8dc
      Jérome Perrin authored
      Business templates are installed by system user, which is a special
      user not returned by getWrappedOwner. Because of this, the "fixing
      problems or activating a disabled alarm is not allowed" error was
      raised when checking if the owner of the alarm has manage portal
      permission on the alarm.
      
      This switches the implementation to explicit creation of the user
      when user id is the system user, so that we have a user with the
      permission to solve the alarm.
      58f6b8dc
  5. 26 May, 2022 5 commits
  6. 24 May, 2022 6 commits
    • Valentin Benozillo's avatar
      erp5_ui_test: Drop the plural · d17614c7
      Valentin Benozillo authored
      d17614c7
    • Valentin Benozillo's avatar
      ui_test: Add range browser error test · 3998056d
      Valentin Benozillo authored
      When start and end are setup on float/integer field, the max and mic attributes are also setup in the input html element.
      So if the user enter a number outside this range, the browser should display an error.
      3998056d
    • Valentin Benozillo's avatar
      erp5_ui_test: Add external validator for float field range check · 789da4e7
      Valentin Benozillo authored
      using start and end filed on float, trigger web-browser error. But UI test are looking for multiple fields check by erp5.
      2 validators script are added because no argument can be passed to external validator, so :
      TALES: python: context.Base_ValidateFloatRange(value, mix=1, max=2)
      doesn't work
      789da4e7
    • Jérome Perrin's avatar
      *: keep using func_code and not yet __code__ with scripts · eaae74a0
      Jérome Perrin authored
      On Zope2, python scripts do not have __code__, they only have
      func_code (and same for __defauls__/func_defaults).
      We tried to backport the support of __code__ from Zope4 as a Zope2
      patch - it was SlapOS patch 4fa33dfc6 (erp5: py3: `func_{code,defaults}`
      was replaced in Python3 by `__{code,defaults}__`., 2022-04-25),
      but this patch was incomplete. We tried to backport more, but then
      realized that we don't need to use __code__ on ERP5 master yet,
      because ERP5 master branch is still supporting Zope2 only.
      
      This patch revert a small part of a17bb910 (py2/py3: Make Products
      code compatible with both python2 and python3., 2022-04-13), the part
      where we use f.__code__ where f might be a python script. For now,
      we'll apply this patch only on the Zope4 branch.
      
      A few places where f.func_code was used and f was a for sure not a
      python script but a simple class method or function are kept here, as
      __code__ support is missing only on in ZODB scripts.
      eaae74a0
    • Jérome Perrin's avatar
      tests: infrastructure to rebuild and export all business templates · 83e69b6b
      Jérome Perrin authored
      This is not really a test, but it reuses runUnitTest/runTestSuite
      commands, because they are good tools to quickly create ERP5
      environment and installing business templates.
      
      To re-build and re-export all* business templates, use this command:
      
          ./bin/runTestSuite --test_suite=ReExportERP5BusinessTemplateTestSuite
      
      --node_quantity argument can also be used to process multiple
      business templates in parallel.
      
      * note that this does not actually handle all business templates, but
      only the ones for which coding style test is enabled, because most
      business templates for which it is not enabled can not be installed.
      
      This typically produces large diffs that should apply the same
      change to many files and ideally, nothing else. We also developed a
      simple tool which summarize the diff by detecting the same chunk
      present in multiple files, it can be found at
      https://lab.nexedi.com/nexedi/erp5/snippets/1171 and also below.
      
      ---
      
      from __future__ import print_function
      """report similar hunks in a patch.
      """
      __version__ = '0.1'
      
      import argparse
      import collections
      import codecs
      import unidiff # unidiff==0.7.3
      import hashlib
      
      parser = argparse.ArgumentParser()
      parser.add_argument('patch_file', type=argparse.FileType('r'), default='-', nargs='?')
      parser.add_argument('-v', '--verbose', action='count', default=0)
      args = parser.parse_args()
      
      patchset = unidiff.PatchSet(codecs.getreader('utf-8')(args.patch_file))
      
      chunks_by_filenames = collections.defaultdict(set)
      
      for patch in patchset:
        for chunk in patch:
          chunk_text = u''.join([unicode(l) for l in chunk])
          chunks_by_filenames[chunk_text].add(patch.path)
      
      for chunk_text, filenames in chunks_by_filenames.items():
        chunk_hash = hashlib.md5(chunk_text.encode('utf-8')).hexdigest()
        print("Chunk %s is present in %s files" % (chunk_hash, len(filenames)))
        if args.verbose:
          print()
          print("\n".join("  " + f for f in sorted(filenames)))
          print()
        if args.verbose > 1:
          print()
          print(chunk_text)
          print()
      83e69b6b
    • Jérome Perrin's avatar
      accounting: always compare rounded values when indexing balance transactions · b075c99b
      Jérome Perrin authored
      To prevent rounding errors, we always compare rounded values to the
      precision of the accounting currency. There was a place here where we
      were using -= without rounding, which caused to detect a difference
      between new and current stock and insert a line for 0 in an existing
      instance for which some accounting lines were created with too precise
      values - but not in a way that was detected by the assertions in
      AccountingPeriod_createBalanceTransaction.
      
      Rounding here as well solved the problem with that data.
      b075c99b
  7. 23 May, 2022 2 commits
  8. 21 May, 2022 1 commit
    • Jérome Perrin's avatar
      ERP5Type/patches: prepare for removal of Products.DCWorkflowGraph · 88321109
      Jérome Perrin authored
      Supports the case where Products.DCWorkflowGraph is not present.
      Even though we are removing Products.DCWorkflowGraph from the
      software release, we don't remove this monkey patch yet, because
      this monkey patch also fixed a severe security issue. We keep the
      patch for the cases where a recent ERP5 runs on an old SlapOS where
      the product is still there.
      
      This change just moves the existing code in a try/except ImportError
      block
      88321109
  9. 19 May, 2022 1 commit
  10. 18 May, 2022 4 commits
  11. 17 May, 2022 5 commits
  12. 16 May, 2022 4 commits
    • Julien Muchembled's avatar
      ERP5Type: fix regression in properties.dtml · eef80b9e
      Julien Muchembled authored
      In commit a17bb910 ("py2/py3:
      Make Products code compatible with both python2 and python3"),
      2to3 changed `_.has_key(...)` to `... in _` whereas _ is not a dict.
      
      Traceback (innermost last):
       ...
       Module OFS.PropertyManager, line 309, in manage_editProperties
         manage_tabs_message=message)
       Module Shared.DC.Scripts.Bindings, line 322, in __call__
         return self._bindAndExec(args, kw, None)
       Module Shared.DC.Scripts.Bindings, line 359, in _bindAndExec
         return self._exec(bound_data, args, kw)
       Module App.special_dtml, line 185, in _exec
         try: result = render_blocks(self._v_blocks, ns)
       Module DocumentTemplate.DT_In, line 707, in renderwob
         try: append(render(section, md))
       Module DocumentTemplate.DT_Let, line 76, in render
         else: d[name]=expr(md)
       Module DocumentTemplate.DT_Util, line 210, in eval
        - __traceback_info__: _
         return eval(code, d)
       Module <string>, line 1, in <module>
      
      TypeError: argument of type 'TemplateDict' is not iterable
      eef80b9e
    • Xiaowu Zhang's avatar
      erp5_trade: add product line selection in stock report · 6cbb159d
      Xiaowu Zhang authored
      See merge request !1628
      6cbb159d
    • Xiaowu Zhang's avatar
      e1360ac0
    • Romain Courteaud's avatar
      erp5_web: force redirection on website using a service worker · 7a7b4a45
      Romain Courteaud authored
      Delete the content of the cache storage (used by erp5js), unregister the service worker and force the browser to reload.
      7a7b4a45
  13. 13 May, 2022 3 commits
    • Romain Courteaud's avatar
      Drop erp5_xhtml_gadget_style · fa144eee
      Romain Courteaud authored
      It is now replaced by erp5_web_renderjs_ui
      fa144eee
    • Jérome Perrin's avatar
      erp5.util: Release erp5.util (0.4.74) · 4774f87a
      Jérome Perrin authored
      4774f87a
    • Jérome Perrin's avatar
      testnode/SlapOSControler: run partitions more times · 6eaf598b
      Jérome Perrin authored
      This is for *StandaloneDeploymentScriptTest* test suites, which
      download a large disk image during slapos node instance step.
      
      For example, on a Wendelin.StandaloneDeploymentScriptTest-Debian.Stretch
      test run [1], we can see that the slapos node instance started at
      2022-05-13 07:50:30,788 and stopped trying at 2022-05-13 07:51:35,885.
      
      In the last run, slapos node instance failed, especially this promise:
      
          2022-05-13 07:51:35,701 INFO     slapgrid_cp: 2022-05-13 07:51:35 slapos[3548917] ERROR ERROR '/srv/slapgrid/slappart22/t/cjl/i/1/srv/monitor/private/virtual-hard-drive-url/virtual-hard-drive-url-processed-config.state' not empty, content available at https://[???]:8026/private/virtual-hard-drive-url/virtual-hard-drive-url-processed-config.state
          ...
      
      and if we access this URL, we can see the error:
      
          [Errno 2] No such file or directory: '/srv/slapgrid/slappart22/t/cjl/i/1/var/virtual-hard-drive-url/update-image-processed.md5sum'
      
      which means that the image is still not downloaded.
      
      testnode still ignore slapos node instance state return code and run
      tests anyway, so the test ran and failed later.
      
      This image is 1.8Go, when I tried to download it locally it took
      56.41 seconds, so it must be generally almost OK.
      
      By retrying up to 60 times, we give more time for the slapos node
      instance step to download the image.
      
      [1]: https://softinst161166.host.vifib.net/cjl-oSgtg7v20D/suite.log
      6eaf598b