Commit cf0782f0 authored by Tim Peters's avatar Tim Peters

Merge ZODB/branches/efge-beforeCommitHook.

This is Florent Guillaume's branch, giving transaction objects
a new beforeCommitHook() method, as proposed by Jim Fulton on
zodb-dev.  Some changes were made to the branch code (comments,
more tests, more words in the docs).
parent 05128835
...@@ -2,6 +2,19 @@ What's new in ZODB3 3.4xx? ...@@ -2,6 +2,19 @@ What's new in ZODB3 3.4xx?
========================== ==========================
Release date: MM-DDD-2005 Release date: MM-DDD-2005
transaction
-----------
Transaction objects have a new method,
``beforeCommitHook(hook, *args, **kws)``. Hook functions registered with
a transaction are called at the start of a top-level commit, before any
of the work is begun, so a hook function can perform any database operations
it likes. See ``test_beforeCommitHook()`` in
``transaction/tests/test_transaction.py`` for a tutorial doctest, and
the ``ITransaction`` interface for details. Thanks to Florent Guillaume
for contributing code and tests.
Tests Tests
----- -----
......
...@@ -100,6 +100,17 @@ Once a subtransaction has been committed, the top-level transaction ...@@ -100,6 +100,17 @@ Once a subtransaction has been committed, the top-level transaction
commit will start with a commit_sub() call instead of a tpc_begin() commit will start with a commit_sub() call instead of a tpc_begin()
call. call.
Before-commit hook
---------------
Sometimes, applications want to execute some code when a transaction is
committed. For example, one might want to delay object indexing until a
transaction commits, rather than indexing every time an object is changed.
Or someone might want to check invariants only after a set of operations. A
pre-commit hook is available for such use cases, just use beforeCommitHook()
passing it a callable and arguments. The callable will be called with its
arguments at the start of the commit (but not for substransaction commits).
Error handling Error handling
-------------- --------------
...@@ -209,6 +220,11 @@ class Transaction(object): ...@@ -209,6 +220,11 @@ class Transaction(object):
# raised, incorporating this traceback. # raised, incorporating this traceback.
self._failure_traceback = None self._failure_traceback = None
# Holds (hook, args, kws) triples added by beforeCommitHook.
# TODO: in Python 2.4, change to collections.deque; lists can be
# inefficient for FIFO access of this kind.
self._before_commit = []
# Raise TransactionFailedError, due to commit()/join()/register() # Raise TransactionFailedError, due to commit()/join()/register()
# getting called when the current transaction has already suffered # getting called when the current transaction has already suffered
# a commit failure. # a commit failure.
...@@ -286,6 +302,9 @@ class Transaction(object): ...@@ -286,6 +302,9 @@ class Transaction(object):
if self.status is Status.COMMITFAILED: if self.status is Status.COMMITFAILED:
self._prior_commit_failed() # doesn't return self._prior_commit_failed() # doesn't return
if not subtransaction:
self._callBeforeCommitHooks()
if not subtransaction and self._sub and self._resources: if not subtransaction and self._sub and self._resources:
# This commit is for a top-level transaction that has # This commit is for a top-level transaction that has
# previously committed subtransactions. Do one last # previously committed subtransactions. Do one last
...@@ -321,6 +340,16 @@ class Transaction(object): ...@@ -321,6 +340,16 @@ class Transaction(object):
self._synchronizers.map(lambda s: s.afterCompletion(self)) self._synchronizers.map(lambda s: s.afterCompletion(self))
self.log.debug("commit") self.log.debug("commit")
def beforeCommitHook(self, hook, *args, **kws):
self._before_commit.append((hook, args, kws))
def _callBeforeCommitHooks(self):
# Call all hooks registered, allowing further registrations
# during processing.
while self._before_commit:
hook, args, kws = self._before_commit.pop(0)
hook(*args, **kws)
def _commitResources(self, subtransaction): def _commitResources(self, subtransaction):
# Execute the two-phase commit protocol. # Execute the two-phase commit protocol.
......
...@@ -107,7 +107,7 @@ class IDataManager(zope.interface.Interface): ...@@ -107,7 +107,7 @@ class IDataManager(zope.interface.Interface):
database is not expected to maintain consistency; it's a database is not expected to maintain consistency; it's a
serious error. serious error.
It's important that the storage calls the passed function It's important that the storage calls the passed function
while it still has its lock. We don't want another thread while it still has its lock. We don't want another thread
to be able to read any updated data until we've had a chance to be able to read any updated data until we've had a chance
to send an invalidation message to all of the other to send an invalidation message to all of the other
...@@ -131,7 +131,7 @@ class IDataManager(zope.interface.Interface): ...@@ -131,7 +131,7 @@ class IDataManager(zope.interface.Interface):
the transaction commits. the transaction commits.
This includes conflict detection and handling. If no conflicts or This includes conflict detection and handling. If no conflicts or
errors occur it saves the objects in the storage. errors occur it saves the objects in the storage.
""" """
def abort(transaction): def abort(transaction):
...@@ -139,8 +139,8 @@ class IDataManager(zope.interface.Interface): ...@@ -139,8 +139,8 @@ class IDataManager(zope.interface.Interface):
Abort must be called outside of a two-phase commit. Abort must be called outside of a two-phase commit.
Abort is called by the transaction manager to abort transactions Abort is called by the transaction manager to abort transactions
that are not yet in a two-phase commit. that are not yet in a two-phase commit.
""" """
def sortKey(): def sortKey():
...@@ -245,6 +245,30 @@ class ITransaction(zope.interface.Interface): ...@@ -245,6 +245,30 @@ class ITransaction(zope.interface.Interface):
# Unsure: is this allowed to cause an exception here, during # Unsure: is this allowed to cause an exception here, during
# the two-phase commit, or can it toss data silently? # the two-phase commit, or can it toss data silently?
def beforeCommitHook(hook, *args, **kws):
"""Register a hook to call before the transaction is committed.
The specified hook function will be called after the transaction's
commit method has been called, but before the commit process has been
started. The hook will be passed the specified positional and keyword
arguments.
Multiple hooks can be registered and will be called in the order they
were registered (first registered, first called). This method can
also be called from a hook: an executing hook can register more
hooks. Applications should take care to avoid creating infinite loops
by recursively registering hooks.
Hooks are called only for a top-level commit. A subtransaction
commit does not call any hooks. If the transaction is aborted, hooks
are not called, and are discarded. Calling a hook "consumes" its
registration too: hook registrations do not persist across
transactions. If it's desired to call the same hook on every
transaction commit, then beforeCommitHook() must be called with that
hook during every transaction; in such a case consider registering a
synchronizer object via a TransactionManager's registerSynch() method
instead.
"""
class IRollback(zope.interface.Interface): class IRollback(zope.interface.Interface):
......
...@@ -631,6 +631,124 @@ def test_join(): ...@@ -631,6 +631,124 @@ def test_join():
""" """
def test_beforeCommitHook():
"""Test the beforeCommitHook.
Let's define a hook to call, and a way to see that it was called.
>>> log = []
>>> def reset_log():
... del log[:]
>>> def hook(arg='no_arg', kw1='no_kw1', kw2='no_kw2'):
... log.append("arg %r kw1 %r kw2 %r" % (arg, kw1, kw2))
Now register the hook with a transaction.
>>> import transaction
>>> t = transaction.begin()
>>> t.beforeCommitHook(hook, '1')
When transaction commit starts, the hook is called, with its
arguments.
>>> log
[]
>>> t.commit()
>>> log
["arg '1' kw1 'no_kw1' kw2 'no_kw2'"]
>>> reset_log()
A hook's registration is consumed whenever the hook is called. Since
the hook above was called, it's no longer registered:
>>> transaction.commit()
>>> log
[]
The hook is only called for a full commit, not for subtransactions.
>>> t = transaction.begin()
>>> t.beforeCommitHook(hook, 'A', kw1='B')
>>> t.commit(subtransaction=True)
>>> log
[]
>>> t.commit()
>>> log
["arg 'A' kw1 'B' kw2 'no_kw2'"]
>>> reset_log()
If a transaction is aborted, no hook is called.
>>> t = transaction.begin()
>>> t.beforeCommitHook(hook, "OOPS!")
>>> transaction.abort()
>>> log
[]
>>> transaction.commit()
>>> log
[]
The hook is called before the commit does anything, so even if the
commit fails the hook will have been called. To provoke failures in
commit, we'll add failing resource manager to the transaction.
>>> class CommitFailure(Exception):
... pass
>>> class FailingDataManager:
... def tpc_begin(self, txn, sub=False):
... raise CommitFailure
... def abort(self, txn):
... pass
>>> t = transaction.begin()
>>> t.join(FailingDataManager())
>>> t.beforeCommitHook(hook, '2')
>>> t.commit()
Traceback (most recent call last):
...
CommitFailure
>>> log
["arg '2' kw1 'no_kw1' kw2 'no_kw2'"]
>>> reset_log()
If several hooks are defined, they are called in order.
>>> t = transaction.begin()
>>> t.beforeCommitHook(hook, '4', kw1='4.1')
>>> t.beforeCommitHook(hook, '5', kw2='5.2')
>>> t.commit()
>>> len(log)
2
>>> log #doctest: +NORMALIZE_WHITESPACE
["arg '4' kw1 '4.1' kw2 'no_kw2'",
"arg '5' kw1 'no_kw1' kw2 '5.2'"]
>>> reset_log()
While executing, a hook can itself add more hooks, and they will all
be called before the real commit starts.
>>> def recurse(txn, arg):
... log.append('rec' + str(arg))
... if arg:
... txn.beforeCommitHook(hook, '-')
... txn.beforeCommitHook(recurse, txn, arg-1)
>>> t = transaction.begin()
>>> t.beforeCommitHook(recurse, t, 3)
>>> transaction.commit()
>>> log #doctest: +NORMALIZE_WHITESPACE
['rec3',
"arg '-' kw1 'no_kw1' kw2 'no_kw2'",
'rec2',
"arg '-' kw1 'no_kw1' kw2 'no_kw2'",
'rec1',
"arg '-' kw1 'no_kw1' kw2 'no_kw2'",
'rec0']
>>> reset_log()
"""
def test_suite(): def test_suite():
from zope.testing.doctest import DocTestSuite from zope.testing.doctest import DocTestSuite
return unittest.TestSuite(( return unittest.TestSuite((
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment