- 18 Jun, 2021 1 commit
-
-
Kirill Smelkov authored
We are already closing Storage on shutdown (see nearby Storage.close call in the patch), but the DB handle was not closed. With classic ZODB it does not really matter in practice, because not closing DB is only leaking RAM resources and the program is anyway terminated soon. However with wendelin.core 2 things are different: in addition to ZODB storage server, there is also synthetic WCFS filesystem from which files are opened and memory-mmaped. In runUnitTest we start both ZODB and WCFS servers and we also shut down them both in the end. The filesystem server can be cleanly unmounted and shutdown only when there are no opened files left on it. Wendelin.core 2 client works by complementing each ZODB connection (zconn) with WCFS-level connection (wconn) to WCFS server. Those two zconn and wconn are kept in sync by wendelin.core 2 client logic: whenever zconn adjusts its view of the database, so does wconn. And whenever zconn is garbage collected, so does wconn is closed to release resources and close corresponding files opened on WCFS. In addition to garbage-collection, wconn is also closed when zconn.db - the ZODB DB handle via which zconn was created - is closed. This is needed to be able to reliably trigger freeing WCFS resources, because even after DB is closed, zconn can still stay alive forever being referenced from some python object - e.g. a frame or traceback or something else. The latter scenario actually happens during runUnitTest run. As the result it leads to inability to unmount and stop WCFS server cleanly: $ ./runUnitTest --load --save --with_wendelin_core -v erp5_core_test:testWendelinCore ... test (erp5.component.test.erp5_version.testWendelinCore.TestWendelinCoreBasic) ... ok ---------------------------------------------------------------------- Ran 1 test in 0.105s OK F0618 19:05:46.359140 35468 wcfs/client/wcfs.cpp:486] CRITICAL: pinner: wcfs /dev/shm/wcfs/b53b61099c740b452b383db6df6dce4ad6d23ba2: wlink25: recvReq: link is down F0618 19:05:46.359173 35468 wcfs/client/wcfs.cpp:487] CRITICAL: wcfs server will likely kill us soon. CRITICAL: pinner: wcfs /dev/shm/wcfs/b53b61099c740b452b383db6df6dce4ad6d23ba2: wlink25: recvReq: link is down CRITICAL: wcfs server will likely kill us soon. Traceback (most recent call last): File ".../bin/runUnitTest", line 312, in <module> sys.exit(runUnitTest.main()) File ".../parts/erp5/Products/ERP5Type/tests/runUnitTest.py", line 926, in main run_only=run_only, File ".../parts/erp5/Products/ERP5Type/tests/runUnitTest.py", line 709, in runUnitTestList wcfs_server.stop() ... File ".../parts/wendelin.core/wcfs/__init__.py", line 543, in _fuse_unmount raise RuntimeError("%s\n(more details logged)" % emsg) RuntimeError: fuse_unmount /dev/shm/wcfs/b53b61099c740b452b383db6df6dce4ad6d23ba2: failed: fusermount: failed to unmount /dev/shm/wcfs/b53b61099c740b452b383db6df6dce4ad6d23ba2: Device or resource busy (more details logged) # logs 2021-06-18 19:05:45.978 INFO root wcfs: unmount/stop wcfs pid32981 @ /dev/shm/wcfs/b53b61099c740b452b383db6df6dce4ad6d23ba2 2021-06-18 19:05:46.068 WARNING root fuse_unmount /dev/shm/wcfs/b53b61099c740b452b383db6df6dce4ad6d23ba2: failed: fusermount: failed to unmount /dev/shm/wcfs/b53b61099c740b452b383db6df6dce4ad6d23ba2: Device or resource busy 2021-06-18 19:05:46.068 WARNING root # lsof /dev/shm/wcfs/b53b61099c740b452b383db6df6dce4ad6d23ba2 2021-06-18 19:05:46.357 WARNING root COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME runUnitTe 32175 slapuser34 24r REG 0,48 111 4 /dev/shm/wcfs/b53b61099c740b452b383db6df6dce4ad6d23ba2/.wcfs/zurl runUnitTe 32175 slapuser34 25u REG 0,48 0 7 /dev/shm/wcfs/b53b61099c740b452b383db6df6dce4ad6d23ba2/head/watch runUnitTe 32175 slapuser34 26r REG 0,48 2097152 9 /dev/shm/wcfs/b53b61099c740b452b383db6df6dce4ad6d23ba2/head/bigfile/00000000000078b4 2021-06-18 19:05:46.358 WARNING root -> kill -TERM wcfs.go ... 2021-06-18 19:05:46.358 WARNING root -> abort FUSE connection ... I've debugged things a bit and even with kirr/ZODB@bbd03b3a ZODB connection stays alive being referenced from some frame objects. -> Fix this problem by explicitly closing ZODB DB nearby tests shutdown before call to wcfs_server.stop
-
- 17 Jun, 2021 1 commit
-
-
Kirill Smelkov authored
This is follow-up to 5796a17a (core_test: Add test to make sure that wendelin.core basically works; nexedi/erp5!1429). In that commit it was said that testWendelinCore "currently passes with wendelin.core 1, which is the default. It also passes as live test with wendelin.core 2. However with wendelin.core 2 it currently fails when run on testnodes ... because we need to amend ERP5 test driver 1. to run tests on a real storage instead of in-RAM Mapping Storage(*), and 2. to spawn WCFS server for each such storage." This patch addresses that latter problem to run testWendelinCore under testnode infrastructure. @rafael and @jerome suggested that we can force a test to be run on a real storage via `runUnitTest --load --save` or via `--activity_node=n`. @rafael also suggested not to generally change the testing driver, but instead make step-by-step progress and first tag each test that uses wendelin.core with an option. Let's go this way now: runUnitTest/custom_zodb are taught to launch WCFS server if wendelin.core usage is requested and software is built with wendelin.core 2. With both changes combined testWendelinCore should now pass OK when run on a testnode with both wendelin.core 1 and wendelin.core 2. This patch is based on a draft patch by @rafael: https://lab.nexedi.com/rafael/erp5/commit/14e3a777. This patch also relies on recent wendelin.core 2 wcfs.py rework which exposed functionality to start WCFS server and to further control it: kirr/wendelin.core@5bfa8cf8. /cc @tomo, @romain, @jerome, @seb
-
- 15 Jun, 2021 3 commits
-
-
Julien Muchembled authored
With software release test suites, there are probably files/folders with permission issues. See commit 380037f8.
-
Jérome Perrin authored
See merge request !1439
-
Jérome Perrin authored
By setting XORG_LOCK_DIR, we can isolate processes when running on slapos test nodes, so that instead of having all test runners used the shared /tmp, which eventually fail with EnvironmentError: All displays locked : [':123', ':124', ':125', ... errors when too many testnodes are currently running or too many testnodes have left some stale lock files. Eventhough since fbdb4ec9 (test: Do not check presence of lockfile in /tmp/ to assume Xvfb is running for this display, 2017-06-07) things have improved, it happens that the lock files are owned by a different unix user (because it was left by another testnode), then Xvfb is not able to remove the stale lock file.
-
- 14 Jun, 2021 2 commits
-
-
Julien Muchembled authored
-
Julien Muchembled authored
This is backward compatibility code for previous commit.
-
- 10 Jun, 2021 1 commit
-
-
Xavier Thompson authored
`inst/test0-0` -> `i/0` See merge request nexedi/erp5!1441
-
- 09 Jun, 2021 2 commits
-
-
Jérome Perrin authored
With bf57228a ([erp5_core] RelationField listbox must also fetch default_params value from the relationfield, 2018-05-09), we introduced a regression that listboxs from relation fields "proxy listbox ids" default params where no longer used. As a result, if we have a listbox which define some catalog search with default parameters, like for example the "Suppliers" or "Clients" listboxs from erp5_trade are doing, these parameters were not taken into account. In the case of trade listbox, this was showing all nodes. This changes Base_getRelatedObjectParameter to get catalog search from the proxy listbox if there is one.
-
Jérome Perrin authored
trade_renderjs_ui_test: tests for "Suppliers", "Clients", "Organisations" and "Persons" Proxy Listbox Ids
-
- 08 Jun, 2021 5 commits
-
-
Jérome Perrin authored
-
Jérome Perrin authored
-
Jérome Perrin authored
This is mostly for editor fields, to fix the problem that components were edited by the text editor and not the code editor. Editor is selected by Base_getEditorFieldPreferredTextEditor, which will use a text editor by default, unless the document has a content type, in which case it will return the most appropriate editor, which is by default the source code editor for documents whose text content is not text/html. It's anyway not wrong to have a content type defined here, Components are TextDocuments and TextDocuments usually have getContentType method.
-
Jérome Perrin authored
See merge request nexedi/erp5!1432
-
Jérome Perrin authored
also enable more plugins: - https://ckeditor.com/cke4/addon/autolink - https://ckeditor.com/cke4/addon/pastebase64 See merge request nexedi/erp5!1428
-
- 07 Jun, 2021 1 commit
-
-
Jérome Perrin authored
-
- 04 Jun, 2021 7 commits
-
-
Jérome Perrin authored
Instead of using an always-different logger name which confuses log analytics system, use the warning category as logger name (or what zLOG.LOG calls "subsystem")
-
Jérome Perrin authored
-
Jérome Perrin authored
Rename this column from "Sale Trade Condition" to "Trade Condition", so that it can be used also for purchase and internal trade conditions.
-
Jérome Perrin authored
- update for ERP5JS compatibility - prevent an error with "Select Payments" update action, when limit was empty - export to generate SEPA credit transfer ( pain.001.001.02 ) - new mode in "Select Payment" to select planned/confirmed payment and validate them at the same time - allow selecting lines individually in "Select Payments" - allow filtering by third party in "Select Payments" (only supported in xhtml_style for now) - fix issue that Deliver transaction leaves PTG in Delivering state - french translations See merge request nexedi/erp5!1400
-
Jérome Perrin authored
Changes following up nexedi/slapos!985 It seems the logo is a bit bigger, or at least a bit different See merge request nexedi/erp5!1424
-
Jérome Perrin authored
When running OCR, we sometimes have issues because processing is "too heavy": - [x] use 2 or 3 Go of disk space for a one page PDF created by erp5_document_scanner, because we convert pdf -> png -> tiff before sending to tesseract. Modern Ghostscript supports running tesseract directly, so we use it if it's available. - [x] use 300% of CPU. Fixed by setting `OMP_THREAD_LIMIT` when running tesseract. This will only apply when OCR from Images. OCR embedded in Ghostscript does not seem to need this. - [x] ... and often crash, so is restarted. This is fixed by updated tesseract. Updates of ghostscript and tesseract are part of nexedi/slapos!985 See merge request nexedi/erp5!1420
-
Jérome Perrin authored
This timestamp break officejs offline capabilities. This reaplies 40f6c8fe ([erp5_xhtml_style] ckeditor: drop hardcoded timestamp parameter in URL, 2017-10-16) on the updated CKEditor This was done with: find product/ERP5/bootstrap/erp5_xhtml_style/SkinTemplateItem/portal_skins/erp5_ckeditor/ckeditor/ -type f | xargs sed -i 's/"L4KA"/""/g' L4KA was the timestamp for this build, that we can in the top of ckeditor.js
-
- 03 Jun, 2021 13 commits
-
-
Jérome Perrin authored
renderjs_ui_trade_zuite is the tests from trade_zuite ported to ERP5JS. Some old tests were not ported yet
-
Jérome Perrin authored
-
Jérome Perrin authored
-
Jérome Perrin authored
Init script just have to print "Data Created." init macro no longer redirect to the order created by Zuite_createSaleOrderForTradeZuite, so that it's more reusable.
-
Jérome Perrin authored
The URL was in the description, so this action was not working since a409a7e6 (Fix jump action by using Base_jumpToRelatedObject to jump and Base_checkPermission to check that the destination module is visible, 2008-12-02)
-
Jérome Perrin authored
-
Jérome Perrin authored
renderjs_ui_pdm_zuite is the tests from pdm_zuite ported to ERP5JS.
-
Jérome Perrin authored
-
Jérome Perrin authored
-
Jérome Perrin authored
Using the same message as in Base_jumpToRelatedObject
-
Jérome Perrin authored
getRelatedValueList seems to have changed in behavior at some point, when passed multiple base_category_list, it only returns document that are member of all the categories. Since this API is inneficient here (it searchs catalog unrestrictedly and then apply check one by one the documents in python), we change to use portal_catalog which is much better suited for this kind of queries.
-
Jérome Perrin authored
By relying on PIL after our monkey-patched OFS.Image.getImageInfo. We keep this monkey-patch for now, because it adds supports to svg See merge request nexedi/erp5!1426
-
Jérome Perrin authored
Since 7f32f8cd (erp5_dms: Add PDF Reader using the pdf.js, 2016-06-24) we have a PDF preview with a javascript PDF view, which is much better way of viewing PDF. This commit made the Thumbnail preview obsolete, also it does not really work on ERP5JS, so remove the thumbnail preview.
-
- 02 Jun, 2021 4 commits
-
-
Julien Muchembled authored
See commit ec3c9cbc.
-
Jérome Perrin authored
-
Jérome Perrin authored
-
Jérome Perrin authored
payment transaction group workflow's deliver action is a two step delivery, which "start delivery" and wait for an interaction to finish the delivery. Add a default interaction workflow which finishes the delivery immediately, otherwise the payment transaction group stay in delivering state when user use the Deliver workflow action.
-