Commit 8afbdd53 authored by Julien Muchembled's avatar Julien Muchembled

client: detect if monkey-patched code changes

parent 54d692c5
...@@ -12,18 +12,25 @@ ...@@ -12,18 +12,25 @@
# #
############################################################################## ##############################################################################
# At the moment, no ZODB release include the following patches.
# Later, this must be replaced by some detection mechanism.
if 1: if 1:
from hashlib import md5
from ZODB.Connection import Connection from ZODB.Connection import Connection
def _check(f, *args):
h = md5(f.func_code.co_code).hexdigest()
assert h in args, h
# Allow serial to be returned as late as tpc_finish # Allow serial to be returned as late as tpc_finish
# #
# This makes possible for storage to allocate serial inside tpc_finish, # This makes possible for storage to allocate serial inside tpc_finish,
# removing the requirement to serialise second commit phase (tpc_vote # removing the requirement to serialise second commit phase (tpc_vote
# to tpc_finish/tpc_abort). # to tpc_finish/tpc_abort).
_check(Connection.tpc_finish,
'f50ed2e5a74f584fa1ecaf1cd70a6b15', # Python 2.6
'ab9b1b8d82c40e5fffa84f7bc4ea3a8b', # Python 2.7
)
def tpc_finish(self, transaction): def tpc_finish(self, transaction):
"""Indicate confirmation that the transaction is done.""" """Indicate confirmation that the transaction is done."""
...@@ -39,6 +46,7 @@ if 1: ...@@ -39,6 +46,7 @@ if 1:
# to be able to read any updated data until we've had a chance # to be able to read any updated data until we've had a chance
# to send an invalidation message to all of the other # to send an invalidation message to all of the other
# connections! # connections!
# <patch>
serial = self._storage.tpc_finish(transaction, callback) serial = self._storage.tpc_finish(transaction, callback)
if serial is not None: if serial is not None:
assert isinstance(serial, str), repr(serial) assert isinstance(serial, str), repr(serial)
...@@ -49,6 +57,7 @@ if 1: ...@@ -49,6 +57,7 @@ if 1:
if obj is not None and obj._p_changed is not None: if obj is not None and obj._p_changed is not None:
obj._p_changed = 0 obj._p_changed = 0
obj._p_serial = serial obj._p_serial = serial
# </patch>
self._tpc_cleanup() self._tpc_cleanup()
Connection.tpc_finish = tpc_finish Connection.tpc_finish = tpc_finish
...@@ -62,6 +71,11 @@ if 1: ...@@ -62,6 +71,11 @@ if 1:
# so this should not be done in afterCompletion(), and anyway, we don't # so this should not be done in afterCompletion(), and anyway, we don't
# know any legitimate use of DB access outside a transaction. # know any legitimate use of DB access outside a transaction.
_check(Connection.afterCompletion,
'70dfc3df8a455d9f663dec619c607eb5', # Python 2.6
'cd3a080b80fd957190ff3bb867149448', # Python 2.7
)
def afterCompletion(self, *ignored): def afterCompletion(self, *ignored):
self._readCurrent.clear() self._readCurrent.clear()
# PATCH: do not call sync() # PATCH: do not call sync()
......
  • Hello @jm,

    Recently with @Tyagov we were trying my wendelin.core tutorial[1] with NEO and when using it out of the box, i.e. pip install wendelin.core neoppod and then

    root = dbopen('neo://dbname@master')

    neo.client.Storage tries to monkey-patch Connection.tpc_finish with first verifying Connection.tpc_finish code is in expected state and "fails":

    >>> import neo.client
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/home/kirr/tmp/trashme/1/local/lib/python2.7/site-packages/neo/client/__
        'ab9b1b8d82c40e5fffa84f7bc4ea3a8b', # Python 2.7
      File "/home/kirr/tmp/trashme/1/local/lib/python2.7/site-packages/neo/client/__
        assert h in args, h
    AssertionError: b7b144f0a4865a272d4200e520193f1d

    this happens e.g. with ZODB 4 in the following configuration:

    (1) kirr@teco:~/tmp/trashme$ pip list
    BTrees (4.3.1)
    futures (3.0.5)
    neoppod (1.6.3)
    persistent (4.2.1)
    pip (8.1.2)
    pkg-resources (0.0.0)
    setuptools (24.0.3)
    six (1.10.0)
    transaction (1.6.1)
    trollius (2.1)
    wheel (0.29.0)
    zc.lockfile (1.2.1)
    ZConfig (3.1.0)
    zdaemon (4.1.0)
    ZEO (5.0.0a0)
    ZODB (4.4.2)
    ZODB3 (3.11.0)
    zodbpickle (0.6.0)
    zope.interface (4.2.0)

    The monkey-patches there in neo/client/__init__.py are for "Allow serial to be returned as late as tpc_finish" and for Connection.afterCompletion not to call sync()

    I see our "Allow serial to be returned as late as tpc_finish" is already upstreamed:

    https://github.com/zopefoundation/ZODB/commit/74fba08a

    and is included in in ZODB >= 4.4.0

    I also see many our patches are already upstreamed too. So my question here is:

    • can we make this monkey-patching conditional and applied only to ZODB3 ?
    • can we expect NEO to reasonably work with ZODB4 (and maybe 5) this way, or there are other blockers?

    I ask because wendelin.core supports both ZODB3 and ZODB4 but for NEO current state is it works only on ZODB3 out-of-the-box.

    Thanks beforehand,
    Kirill

    [1] https://www.nexedi.com/wendelin-Core.Tutorial.2016

  • See the 'next' branch: I have already started to work on this. I am also impatient to make ERP5 & NEO use more recent versions of ZODB (another blocker for this is erp5!112 (merged)).

    I prefer to finish some development on ZODB5 (remove support of old commit protocol) before anything else.

    software/neoppod/software-zodb4.cfg is pinned to version 4.3.1. Can't you do the same for a while ?

  • @kirr , I suggest we go the way of fixing tutorial for now so it becomes usable.

  • @jm thanks for feedback. I already saw the next branch and was just curious about current status. Btw it feels great that ZODB is being revived this days. If there were no ZODB3 / ZODB eggs split we could just limit neoppod to require ZODB <= 4.4.0dev, but with the split and current setuptools state such limiting is not possible.

    Thus I've adjusted the tutorial to use ZODB 4.3.X

    Good luck with ZODB5 and thanks again,
    Kirill

Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment