• Kirill Smelkov's avatar
    persistent: On deactivate release in-slots objects too · 9ddbacb0
    Kirill Smelkov authored
    ( This is backport of https://github.com/zopefoundation/persistent/pull/44
      to ZODB-3.10 )
    
    On ._p_deactivate() and ._p_invalidate(), when an object goes to ghost
    state, objects referenced by all its attributes, except related to
    persistence machinery, are released, this way freeing memory (if they
    were referenced only from going-to-ghost object).
    
    That's the idea - an object in ghost state is simply a stub, which loads
    its content on first access (via hooking into get/set attr) while
    occupying minimal memory in not-yet-loaded state.
    
    However the above is not completely true right now, as currently on
    ghostification only object's .__dict__ is released, while in-slots objects
    are retained attached to ghost object staying in RAM:
    
        ---- 8< ----
        from ZODB import DB
        from persistent import Persistent
        import gc
    
        db = DB(None)
        jar = db.open()
    
        class C:
            def __init__(self, v):
                self.v = v
            def __del__(self):
                print 'released (%s)' % self.v
    
        class P1(Persistent):
            pass
    
        class P2(Persistent):
            __slots__ = ('aaa')
    
        p1 = P1()
        jar.add(p1)
        p1.aaa = C(1)
    
        p2 = P2()
        jar.add(p2)
        p2.aaa = C(2)
    
        p1._p_invalidate()
        # "released (1)" is printed
    
        p2._p_invalidate()
        gc.collect()
        # "released (2)" is NOT printed     <--
        ---- 8< ----
    
    So teach ghostify() & friends to release objects in slots to free-up
    memory when an object goes to ghost state.
    
    NOTE PyErr_Occurred() added after ghostify() calls because
    pickle_slotnames() can raise an error, but we do not want to change
    ghostify() prototype for backward compatibility reason - as it is used
    in cPersistenceCAPIstruct.
    
    ( I hit this bug with wendelin.core which uses proxies to load
      data from DB to virtual memory manager and then deactivate proxy right
      after load has been completed:
    
      https://lab.nexedi.com/nexedi/wendelin.core/blob/f7803634/bigfile/file_zodb.py#L239
      https://lab.nexedi.com/nexedi/wendelin.core/blob/f7803634/bigfile/file_zodb.py#L295 )
    9ddbacb0
testPersistent.py 9.95 KB