- 13 Sep, 2016 4 commits
-
-
Julien Muchembled authored
-
Kirill Smelkov authored
( This is backport of https://github.com/zopefoundation/persistent/pull/44 to ZODB-3.10 ) On ._p_deactivate() and ._p_invalidate(), when an object goes to ghost state, objects referenced by all its attributes, except related to persistence machinery, are released, this way freeing memory (if they were referenced only from going-to-ghost object). That's the idea - an object in ghost state is simply a stub, which loads its content on first access (via hooking into get/set attr) while occupying minimal memory in not-yet-loaded state. However the above is not completely true right now, as currently on ghostification only object's .__dict__ is released, while in-slots objects are retained attached to ghost object staying in RAM: ---- 8< ---- from ZODB import DB from persistent import Persistent import gc db = DB(None) jar = db.open() class C: def __init__(self, v): self.v = v def __del__(self): print 'released (%s)' % self.v class P1(Persistent): pass class P2(Persistent): __slots__ = ('aaa') p1 = P1() jar.add(p1) p1.aaa = C(1) p2 = P2() jar.add(p2) p2.aaa = C(2) p1._p_invalidate() # "released (1)" is printed p2._p_invalidate() gc.collect() # "released (2)" is NOT printed <-- ---- 8< ---- So teach ghostify() & friends to release objects in slots to free-up memory when an object goes to ghost state. NOTE PyErr_Occurred() added after ghostify() calls because pickle_slotnames() can raise an error, but we do not want to change ghostify() prototype for backward compatibility reason - as it is used in cPersistenceCAPIstruct. ( I hit this bug with wendelin.core which uses proxies to load data from DB to virtual memory manager and then deactivate proxy right after load has been completed: https://lab.nexedi.com/nexedi/wendelin.core/blob/f7803634/bigfile/file_zodb.py#L239 https://lab.nexedi.com/nexedi/wendelin.core/blob/f7803634/bigfile/file_zodb.py#L295 )
-
Julien Muchembled authored
-
Julien Muchembled authored
This reverts to the behaviour of 3.10.3 and older. (cherry picked from commit b74eef76) Conflicts: src/ZODB/tests/testconflictresolution.py
-
- 28 Apr, 2016 2 commits
-
-
Julien Muchembled authored
-
Julien Muchembled authored
Multi-threaded IO support, which is new to ZODB 3.10, allows clients to read data (load & loadBefore) even after tpc_vote has started to write a new transaction to disk. This is done by using different 'file' objects. Issues start when a transaction is rolled back after data has been appended (using the writing file object). Truncating is not enough because the FilePool may have been used concurrently to read the end of the last transaction: file objects have their own read buffers which, in this case, may also contain the beginning of the aborted transaction. So a solution is to invalidate read buffers whenever they may contain wrong data. This patch does it on truncation, which happens rarely enough to not affect performance. We discovered this bug in the following conditions: - ZODB splitted in several FileStorage - many conflicts in the first committed DB, but always resolved - unresolved conflict in another DB If the transaction is replayed with success (no more conflict in the other DB), a subsequent load of the object that could be resolved in the first DB may, for example, return a wrong serial (tid of the aborted transaction) if the layout of the committed transaction matches that of the aborted one. The bug usually manifests with POSKeyError & CorruptedDataError exceptions in ZEO logs, for example while trying to resolve a conflict (and restarting the transaction does not help, causing Site Errors in Zope). But theorically, this could also cause silent corruption or unpickling errors at client side. (cherry picked from commit 028b1922) Conflicts: src/ZODB/FileStorage/FileStorage.py
-
- 17 Mar, 2016 4 commits
-
-
Tres Seaver authored
Fixing uncaught exception problem on shutdown
-
Patrick Gerken authored
-
Patrick Gerken authored
-
Patrick Gerken authored
-
- 16 Mar, 2016 1 commit
-
-
Patrick Gerken authored
Nested connections try to clean up themselves multiple times. This does not work
-
- 10 Mar, 2014 3 commits
-
-
Godefroid Chapelle authored
-
Godefroid Chapelle authored
-
Gediminas Paulauskas authored
-
- 03 Mar, 2014 2 commits
-
-
Godefroid Chapelle authored
-
Godefroid Chapelle authored
-
- 03 Feb, 2014 6 commits
-
-
Godefroid Chapelle authored
-
Godefroid Chapelle authored
-
Godefroid Chapelle authored
-
-
Godefroid Chapelle authored
-
-
- 23 Dec, 2013 1 commit
-
-
Godefroid Chapelle authored
-
- 21 Oct, 2013 2 commits
-
-
Godefroid Chapelle authored
when objects had been added in readCurrent still need to come up with a test
-
Godefroid Chapelle authored
-
- 17 Sep, 2013 3 commits
-
-
Godefroid Chapelle authored
-
Godefroid Chapelle authored
-
Godefroid Chapelle authored
-
- 11 Jun, 2012 1 commit
-
-
Tres Seaver authored
-
- 25 May, 2012 1 commit
-
-
Tres Seaver authored
-
- 19 May, 2012 2 commits
-
-
Jim Fulton authored
-
Jim Fulton authored
string representation.
-
- 02 May, 2012 3 commits
-
-
-
Tres Seaver authored
-
Tres Seaver authored
-
- 19 Nov, 2011 1 commit
-
-
Jim Fulton authored
-
- 18 Nov, 2011 1 commit
-
-
Jim Fulton authored
Conflict resolution failed when state included cross-database persistent references with classes that couldn't be imported.
-
- 17 Nov, 2011 2 commits
-
-
Jim Fulton authored
-
Jim Fulton authored
the fix fix.
-
- 16 Nov, 2011 1 commit
-
-
Jim Fulton authored
Conflict resolution failed when state included persistent references with classes that couldn't be imported.
-