- 05 Aug, 2021 1 commit
-
-
Kirill Smelkov authored
This patch is similar to nexedi/erp5@530e8b4e (nexedi/erp5!1445) and amends Wendelin test driver to run tests on a real storage and spawn WCFS as needed. Running tests on a real storage instead of in-RAM MappingStorage is required in wendelin.core 2 case, because WCFS and Zope need to access the same ZODB, and if Zope keeps data inside in-RAM MappingStorage, there is no easy way for WCFS to access it. In such a case tests are failing as e.g. ERROR: test_03_DataArray (erp5.component.test.erp5_version.testWendelin.Test) ---------------------------------------------------------------------- Traceback (most recent call last): File "<portal_components/test.erp5.testWendelin>", line 185, in test_03_DataArray self.tic() File "/srv/slapgrid/slappart15/srv/runner/software/f0bddbb3930203c84c98ece4787c4df6/parts/erp5/product/ERP5Type/tests/ProcessingNodeTestCase.py", line 309, in tic raise RuntimeError(error_message) ... Last error message: ValueError ZODB.MappingStorage.MappingStorage is in-RAM storage in-RAM storages are not supported: a zurl pointing to in-RAM storage in one process would lead to another in-RAM storage in WCFS process. Traceback (innermost last): Module Products.CMFActivity.ActivityTool, line 361, in __call__ result = method(*self.args, **self.kw) Module Products.ERP5Type.patches.PythonScript, line 179, in __call__ return self._orig_bindAndExec(args, kw, None) Module Shared.DC.Scripts.Bindings, line 359, in _bindAndExec return self._exec(bound_data, args, kw) Module Products.ERP5Type.tests.ERP5TypeTestCase, line 1576, in _exec return PythonScript_exec(self, *args) Module Products.PythonScripts.PythonScript, line 344, in _exec result = f(*args, **kw) Module script, line 21, in DataStream_readChunkListAndTransform - <PythonScript at /erp5_portal_30347777537315e139b1afecf8479bcc/DataStream_readChunkListAndTransform used for /erp5_portal_30347777537315e139b1afecf8479bcc/data_stream_module/1> - Line 21 **kw) Module AccessControl.ZopeGuards, line 369, in guarded_apply return builtin_guarded_apply(func, args, kws) Module AccessControl.ZopeGuards, line 391, in builtin_guarded_apply return func(*arglist, **argdict) Module Products.ERP5Type.patches.ExternalMethod, line 114, in __call__ return _f[0](*args, **kw) Module erp5.component.extension.erp5_version.Wendelin, line 58, in DataStream_copyCSVToDataArray zarray[-ndarray_shape[0]:] = ndarray Module wendelin.bigarray, line 444, in __setitem__ a = self.__getitem__(idx) Module wendelin.bigarray, line 411, in __getitem__ vmaM = self._fileh.mmap(pageM_min, pageM_max-pageM_min+1) Module wendelin.bigarray.array_zodb, line 94, in _fileh self._v_fileh = self.zfile.fileh_open() Module wendelin.bigfile.file_zodb, line 628, in fileh_open fileh = _ZBigFileH(self, _use_wcfs) Module wendelin.bigfile.file_zodb, line 689, in __init__ self.zfileh = zfile._v_file.fileh_open(use_wcfs) Module wendelin.bigfile._file_zodb, line 112, in wendelin.bigfile._file_zodb._ZBigFile.fileh_open pywconn = wczsync.pywconnOf(zconn) Module wendelin.wcfs.client._wczsync, line 54, in wendelin.wcfs.client._wczsync.pywconnOf zurl = pyzodb.zstor_2zurl(zstor) Module wendelin.lib.zodb, line 338, in zstor_2zurl return "demo:(%s)/(%s)" % (zstor_2zurl(zstor.base), zstor_2zurl(zstor.changes)) Module wendelin.lib.zodb, line 403, in zstor_2zurl "\tanother in-RAM storage in WCFS process.") % ztype) ValueError: ZODB.MappingStorage.MappingStorage is in-RAM storage in-RAM storages are not supported: a zurl pointing to in-RAM storage in one process would lead to another in-RAM storage in WCFS process. -> Fix it by using a real storage for tests via --load/--save hack, and via --with_wendelin_core requesting to spawn WCFS when built with wendelin.core 2. Please see more details in added comment and in nexedi/erp5@530e8b4e. Test results: - Wendelin/WC1: https://nexedijs.erp5.net/#/test_result_module/20210728-55E6A5B6 - Wendelin/WC2: https://nexedijs.erp5.net/#/test_result_module/20210729-A15AF293 /cc @tomo, @rafael, @Tyagov, @klaus, @romain, @jerome, @seb, @jp (reminder to review all tests to run on a real storage) /reviewed-on nexedi/wendelin!96
-
- 14 Jun, 2021 1 commit
-
-
Ivan Tyagov authored
See merge request nexedi/wendelin!95
-
- 11 Jun, 2021 1 commit
-
-
Roque authored
- fix ebulk install command for ios - update embulk manual installation url
-
- 26 May, 2021 3 commits
-
-
Ivan Tyagov authored
See merge request nexedi/wendelin!94
-
Ivan Tyagov authored
-
Ivan Tyagov authored
-
- 25 May, 2021 1 commit
-
-
Ivan Tyagov authored
-
- 21 May, 2021 2 commits
-
-
Ivan Tyagov authored
-
Ivan Tyagov authored
-
- 27 Apr, 2021 1 commit
-
-
Ivan Tyagov authored
See merge request nexedi/wendelin!93
-
- 26 Apr, 2021 1 commit
-
-
Roque authored
-
- 18 Mar, 2021 3 commits
-
-
Ivan Tyagov authored
-
Ivan Tyagov authored
-
Ivan Tyagov authored
-
- 17 Mar, 2021 1 commit
-
-
Ivan Tyagov authored
-
- 08 Mar, 2021 8 commits
-
-
Roque authored
See merge request nexedi/wendelin!92
-
Roque authored
-
Roque authored
-
Roque authored
See merge request nexedi/wendelin!91
-
Roque authored
-
Roque authored
-
Roque authored
-
Roque authored
-
- 05 Mar, 2021 4 commits
-
-
Ivan Tyagov authored
See merge request nexedi/wendelin!89
-
Roque authored
See merge request nexedi/wendelin!90
-
Roque authored
-
Eteri authored
-
- 04 Mar, 2021 4 commits
-
-
Ivan Tyagov authored
See merge request nexedi/wendelin!88
-
Roque authored
erp5_wendelin_data_lake_ingestion_default_security_model: add roles to Notebook and Knowledge Pad modules
-
Ivan Tyagov authored
See merge request nexedi/wendelin!87
-
Eteri authored
-
- 03 Mar, 2021 2 commits
-
-
Roque authored
See merge request nexedi/wendelin!86
-
Roque authored
-
- 02 Mar, 2021 6 commits
-
-
Ivan Tyagov authored
See merge request nexedi/wendelin!81
-
Roque authored
-
Roque authored
-
Roque authored
-
Roque authored
queries - new script to get data set number of files
-
Roque authored
-
- 24 Feb, 2021 1 commit
-
-
Ivan Tyagov authored
erp5_wendelin_data_lake: add an utility script to check md5sum of all Data Streams for a Data Set (uploaded from file system) and what we have in reallity.
-