- 03 May, 2021 1 commit
-
-
Kirill Smelkov authored
-
- 26 Mar, 2021 19 commits
-
-
Kirill Smelkov authored
* master: go/*: Cosmetics go/xzodb: New package that provides zodb-related utilities shared by drivers (draft) go/zodb/demo: New package that provides base+δ storages overlay go/zodb: Provide OpenDriver in addition to Open go/zodb/fs1: Report URL with file:// schema included go/zodb: Fix Open to detect lack of schema only by ":" instead of by "://" go/zodb: Fix IStorage implementation not to deadlock driver if watcher fails go/zodb/zeo: Fix it to provide "Close vs watchq" guaranty go/zodb/fs1: Fix it to provide "Close vs watchq" guaranty go/zodb: Require drivers to close watchq on Close go/internal/xtesting: Fix FatalIf to return a helper
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
For Load demo.Storage implementation is similar to DemoStorage in ZODB/py with fixes "cherry-picked" from: - https://github.com/zopefoundation/ZODB/issues/318 (DemoStorage does not take whiteouts into account -> leading to data corruption) - https://github.com/zopefoundation/ZODB/pull/323 (loadAt + fix for the above issue) For safety demo.Storage - contrary to DemoStorage/py - actually verifies that for demo=base+δ δ comes strictly after base and that base remains unchanged. URI schema follows XRI Cross-references approach and is demo:(zurl_base)/(zurl_δ) https://en.wikipedia.org/wiki/Extensible_Resource_Identifier provides some related details and examples. For ZODB/py corresponding pull-request for zodburi to support demo: URI scheme has been made here: https://github.com/Pylons/zodburi/pull/29 . Tests need: - recent zodbtools with zodbrestore: https://lab.nexedi.com/nexedi/zodbtools/blob/129afa67/zodbtools/zodbrestore.py nexedi/zodbtools!19 - ZODB with support for DemoStorage.deleteObject https://github.com/zopefoundation/ZODB/pull/341 On Go side demo storage is needed for wendelin.core 2 because ERP5 uses DemoStorage to run tests.
-
Kirill Smelkov authored
This is low-level API to open IStorageDriver instead of IStorage. Demo storage will need this. Maybe it would be a good idea to move drivers-related functionality into separate package in the future.
-
Kirill Smelkov authored
In ZODB/go when there is no schema in zurl, open automatically prepends file:// . However filename itself could contain ":" and so generally speaking it is incorrect to return URL without file:// schema prepended to file name. Another reason to always use fully-constructed URLs with schema, is interoperability with ZODB/py - there zodburi, when given zurl without schema, does not make any assumption that it is of file:// kind and rejects opening such URIs.
-
Kirill Smelkov authored
An URI schema is required to have ":" after it, but - even if frequently used in practice - not //. We will soon introduce "demo:" URI scheme that comes without //, so fix Open to detect schema presence just by ":" and not to fixup "demo:..." url to "file://demo:..." automatically.
-
Kirill Smelkov authored
Before the patch if storage.watcher fails, storage.driver.Close is not called, and so the driver will continue to send to .drvWatchq, but noone is receiving from it. a5dbb92b (go/zodb: Require drivers to close watchq on Close), provides the guarantee that the driver will stop sending on drvWatchq right after drv.Close call.
-
Kirill Smelkov authored
Provide guaranty that Close forces the driver to stop sending to watchq and to close it. See a5dbb92b ("go/zodb: Require drivers to close watchq on Close") for details. Without the fix TestWatch fails with test timeout: panic: test timed out after 30s # Close waits for serve to stop goroutine 93 [semacquire]: sync.runtime_Semacquire(0xc000152170) /home/kirr/src/tools/go/go/src/runtime/sema.go:56 +0x45 sync.(*WaitGroup).Wait(0xc000152168) /home/kirr/src/tools/go/go/src/sync/waitgroup.go:130 +0x65 lab.nexedi.com/kirr/neo/go/zodb/storage/zeo.(*zLink).Close(0xc0001520f0, 0x1313, 0x1) /home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/zodb/storage/zeo/zrpc.go:159 +0x47 lab.nexedi.com/kirr/neo/go/zodb/storage/zeo.(*zeo).Close(0xc000313680, 0xc000107c78, 0x1) /home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/zodb/storage/zeo/zeo.go:526 +0x2e lab.nexedi.com/kirr/neo/go/internal/xtesting.DrvTestWatch(0xc000082c00, 0xc0000aa2a0, 0x24, 0x6a4a38) /home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/internal/xtesting/xtesting.go:442 +0xdb5 lab.nexedi.com/kirr/neo/go/zodb/storage/zeo.TestWatch.func1(0xc000082c00, 0x6e3498, 0xc00009a380) /home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/zodb/storage/zeo/zeo_test.go:270 +0x99 lab.nexedi.com/kirr/neo/go/zodb/storage/zeo.withZEOSrv.func2.1(0xc0000a4168, 0x16) /home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/zodb/storage/zeo/zeo_test.go:207 +0xfb lab.nexedi.com/kirr/neo/go/zodb/storage/zeo.withZEOSrv.func1(0xc000082c00, 0xc00009c180) /home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/zodb/storage/zeo/zeo_test.go:186 +0x129 lab.nexedi.com/kirr/neo/go/zodb/storage/zeo.withZEOSrv.func2(0xc000082c00) /home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/zodb/storage/zeo/zeo_test.go:199 +0x10e testing.tRunner(0xc000082c00, 0xc00009c160) /home/kirr/src/tools/go/go/src/testing/testing.go:1194 +0xef created by testing.(*T).Run /home/kirr/src/tools/go/go/src/testing/testing.go:1239 +0x2b3 # serve is stuck in invalidateTransaction doing watchq<- goroutine 26 [chan send]: lab.nexedi.com/kirr/neo/go/zodb/storage/zeo.(*zeo).invalidateTransaction(0xc000313680, 0x6417e0, 0xc000323b60, 0x0, 0x0) /home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/zodb/storage/zeo/zeo.go:176 +0x373 lab.nexedi.com/kirr/neo/go/zodb/storage/zeo.(*zLink).serveRecv1(0xc0001520f0, 0xc000393890, 0x0, 0x0) /home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/zodb/storage/zeo/zrpc.go:225 +0x4b4 lab.nexedi.com/kirr/neo/go/zodb/storage/zeo.(*zLink).serveRecv(0xc0001520f0) /home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/zodb/storage/zeo/zrpc.go:176 +0x8d created by lab.nexedi.com/kirr/neo/go/zodb/storage/zeo.(*zLink).start /home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/zodb/storage/zeo/zrpc.go:99 +0xc8
-
Kirill Smelkov authored
Provide guaranty that Close forces the driver to stop sending to watchq and to close it. See a5dbb92b ("go/zodb: Require drivers to close watchq on Close") for details.
-
Kirill Smelkov authored
If we don't require drivers to stop sending to watchq after Close, there could be many deadlock scenarios, for example: - client called drv.Close(); no longer listens to watchq; driver is stuck sending to watchq, or - client called drv.Close(), which itself waits for tasks spawned inside driver to complete, which are stuck sending to watchq because client no longer does <-watchq. The change is similar in spirit to safety guaranty provided by high-level Watcher where after DelWatch call it is guaranteed that there will be no more sends to subscribed watchq (see c41c2907 "go/zodb: High-level watching - initial draft") for details. All drivers don't provide requested guarantee yet. We'll be fixing them one-by-one in followup commits.
-
Kirill Smelkov authored
Else, on an error, it is the lineno of `t.Fatal(err)` inside FatalIf that is printed, not the line number inside user test.
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Julien Muchembled authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
- 25 Mar, 2021 1 commit
-
-
Kirill Smelkov authored
ZEO and NEO are not yet adjusted to new "watchq vs Close" requirement.
-
- 22 Mar, 2021 1 commit
-
-
Kirill Smelkov authored
For https://github.com/DataDog/czlib/pull/5
-
- 21 Mar, 2021 1 commit
-
-
Kirill Smelkov authored
-
- 20 Mar, 2021 12 commits
-
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
- 17 Mar, 2021 5 commits
-
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-