Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Z
ZODB
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Analytics
Analytics
CI / CD
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
nexedi
ZODB
Commits
8f33e101
Commit
8f33e101
authored
Sep 12, 2016
by
Jim Fulton
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Many changes based on excellent PR comments.
More changes to come.
parent
811403fc
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
40 additions
and
33 deletions
+40
-33
doc/guide/transactions-and-threading.rst
doc/guide/transactions-and-threading.rst
+40
-33
No files found.
doc/guide/transactions-and-threading.rst
View file @
8f33e101
...
@@ -6,7 +6,7 @@ Transactions and concurrency
...
@@ -6,7 +6,7 @@ Transactions and concurrency
`Transactions <https://en.wikipedia.org/wiki/Database_transaction>`_
`Transactions <https://en.wikipedia.org/wiki/Database_transaction>`_
are a core feature of ZODB. Much has been written about transactions,
are a core feature of ZODB. Much has been written about transactions,
and we won't go into much detail here. Transactions provide
2
core
and we won't go into much detail here. Transactions provide
two
core
benefits:
benefits:
Atomicity
Atomicity
...
@@ -21,7 +21,7 @@ Concurrency
...
@@ -21,7 +21,7 @@ Concurrency
Transactions provide a way of managing concurrent updates to data.
Transactions provide a way of managing concurrent updates to data.
Different programs operate on the data independently, without having
Different programs operate on the data independently, without having
to use low-level techniques to moderate their access. Coordination
to use low-level techniques to moderate their access. Coordination
and synchronization happen
s
via transactions.
and synchronization happen via transactions.
.. _using-transactions-label:
.. _using-transactions-label:
...
@@ -51,7 +51,7 @@ If we decide we don't want to commit a transaction, we can use
...
@@ -51,7 +51,7 @@ If we decide we don't want to commit a transaction, we can use
``abort``::
``abort``::
conn.root.x = 2
conn.root.x = 2
transaction.abort()
transaction.abort()
# conn.root.x goes back to 1
.. -> src
.. -> src
...
@@ -68,8 +68,8 @@ explanation. When using transactions, there are three kinds of
...
@@ -68,8 +68,8 @@ explanation. When using transactions, there are three kinds of
objects involved:
objects involved:
Transaction
Transaction
Transactions represent units of work.
They have beginnings
and
Transactions represent units of work.
Each transaction has a beginning
and
ends. They
provide the
an end. Transaction
provide the
:interface:`~transaction.interfaces.ITransaction` interface.
:interface:`~transaction.interfaces.ITransaction` interface.
Transaction manager
Transaction manager
...
@@ -83,7 +83,7 @@ Transaction manager
...
@@ -83,7 +83,7 @@ Transaction manager
Data manager
Data manager
Data managers manage data associated with transactions. ZODB
Data managers manage data associated with transactions. ZODB
connections are data managers. The details of how they interact
connections are data managers. The details of how they interact
with transactions
is
n't important here.
with transactions
are
n't important here.
Explicit transaction managers
Explicit transaction managers
-----------------------------
-----------------------------
...
@@ -91,7 +91,7 @@ Explicit transaction managers
...
@@ -91,7 +91,7 @@ Explicit transaction managers
ZODB connections have transaction managers associated with them when
ZODB connections have transaction managers associated with them when
they're opened. When we call the database :meth:`~ZODB.DB.open` method
they're opened. When we call the database :meth:`~ZODB.DB.open` method
without an argument, a thread-local transaction manager is used. Each
without an argument, a thread-local transaction manager is used. Each
thread has it
'
s own transaction manager. When we called
thread has its own transaction manager. When we called
``transaction.commit()`` above we were calling commit on the
``transaction.commit()`` above we were calling commit on the
thread-local transaction manager.
thread-local transaction manager.
...
@@ -99,15 +99,16 @@ Because we used a thread-local transaction manager, all of the work in
...
@@ -99,15 +99,16 @@ Because we used a thread-local transaction manager, all of the work in
the transaction needs to happen in the same thread. Similarly, only
the transaction needs to happen in the same thread. Similarly, only
one transaction can be active in a thread.
one transaction can be active in a thread.
If we want to have transactions who's work is spread over multiple
If we want to run multiple simultaneous transactions in a single
threads, or if we wanted to run multiple simultaneous transactions in
thread, or if we want to spread the work of a transaction over
a single thread, then we can create transaction managers ourselves and
multiple threads [#bad-idea-using-multiple-threads-per-transaction]_,
pass them to :meth:`~ZODB.DB.open`::
then we can create transaction managers ourselves and pass them to
:meth:`~ZODB.DB.open`::
tm
= transaction.TransactionManager()
my_transaction_manager
= transaction.TransactionManager()
conn = db.open(
tm
)
conn = db.open(
my_transaction_manager
)
conn.root.x = 2
conn.root.x = 2
tm
.commit()
my_transaction_manager
.commit()
.. -> src
.. -> src
...
@@ -116,17 +117,17 @@ pass them to :meth:`~ZODB.DB.open`::
...
@@ -116,17 +117,17 @@ pass them to :meth:`~ZODB.DB.open`::
In this example, to commit our work, we called ``commit()`` on the
In this example, to commit our work, we called ``commit()`` on the
transaction manager we created and passed to :meth:`~ZODB.DB.open`.
transaction manager we created and passed to :meth:`~ZODB.DB.open`.
c
ontext managers
C
ontext managers
----------------
----------------
In the examples above, the transaction beginnings were
In the examples above, the transaction beginnings were
implicit. Transaction
'
s were effectively
implicit. Transactions were effectively
[#implicit-transaction-creation]_ created when the transaction
[#implicit-transaction-creation]_ created when the transaction
managers were created and when previous transactions were committed.
managers were created and when previous transactions were committed.
We can create transactions explicitly using
We can create transactions explicitly using
:meth:`~transaction.interfaces.ITransactionManager.begin`::
:meth:`~transaction.interfaces.ITransactionManager.begin`::
tm
.begin()
my_transaction_manager
.begin()
.. -> src
.. -> src
...
@@ -137,7 +138,7 @@ boundaries is to use context managers and the Python ``with``
...
@@ -137,7 +138,7 @@ boundaries is to use context managers and the Python ``with``
statement. Transaction managers are context managers, so we can use
statement. Transaction managers are context managers, so we can use
them with the ``with`` statement directly::
them with the ``with`` statement directly::
with
tm
as trans:
with
my_transaction_manager
as trans:
trans.note("incrementing x")
trans.note("incrementing x")
conn.root.x += 1
conn.root.x += 1
...
@@ -209,7 +210,7 @@ This is because it's still in the same transaction that was implicitly
...
@@ -209,7 +210,7 @@ This is because it's still in the same transaction that was implicitly
begun when a change was last committed against it. If we want to see
begun when a change was last committed against it. If we want to see
changes, we have to begin a new transaction:
changes, we have to begin a new transaction:
>>> trans =
tm
.begin()
>>> trans =
my_transaction_manager
.begin()
>>> conn.root.x
>>> conn.root.x
5
5
...
@@ -232,7 +233,7 @@ connections will get a conflict error when it tries to commit::
...
@@ -232,7 +233,7 @@ connections will get a conflict error when it tries to commit::
conn2.root.x += 1
conn2.root.x += 1
conn.root.x = 9
conn.root.x = 9
tm
.commit() # will raise a conflict error
my_transaction_manager
.commit() # will raise a conflict error
.. -> src
.. -> src
...
@@ -246,7 +247,7 @@ last line. After a conflict error is raised, we'd need to abort the
...
@@ -246,7 +247,7 @@ last line. After a conflict error is raised, we'd need to abort the
transaction, or begin a new one, at which point we'd see the data as
transaction, or begin a new one, at which point we'd see the data as
written by the other connection:
written by the other connection:
>>>
tm
.abort()
>>>
my_transaction_manager
.abort()
>>> conn.root.x
>>> conn.root.x
6
6
...
@@ -260,26 +261,24 @@ This isn't always easy.
...
@@ -260,26 +261,24 @@ This isn't always easy.
Sometimes you may need to queue some operations that update shared
Sometimes you may need to queue some operations that update shared
data structures, like indexes, so the updates can be made by a
data structures, like indexes, so the updates can be made by a
dedicated thread or process.
dedicated thread or process
, without simultaneous updates
.
Conflict resolution
Conflict resolution
~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~
ZODB provides a conflict-resolution framework for merging conflicting
ZODB provides a conflict-resolution framework for merging conflicting
changes. This is implemented by `BTree
changes. Commonly used objects that implement conflict resolution are
<https://pythonhosted.org/BTrees/>`_ buckets and ``Length`` objects.
buckets and ``Length`` objects provided by the `BTree
<https://pythonhosted.org/BTrees/>`_ package.
The main data structures provided by BTrees: BTrees and TreeSets,
The main data structures provided by BTrees: BTrees and TreeSets,
spread their data over multiple objects. The leaf-level objects,
spread their data over multiple objects. The leaf-level objects,
called *buckets* allow distinct keys to be updated without causing
called *buckets* allow distinct keys to be updated without causing
conflicts [#usually-avoids-conflicts]_.
conflicts [#usually-avoids-conflicts]_.
``Length`` objects are conflict
key
counters, that merge changes by
``Length`` objects are conflict
-free
counters, that merge changes by
simply accumulating changes.
simply accumulating changes.
The use of BTree buckets, and to a lesser degree ``Length`` objects
is a very common technique.
.. caution::
.. caution::
Conflict resolution weakens consistency. Resist the temptation to
Conflict resolution weakens consistency. Resist the temptation to
try to implement conflict resolution yourself. In the future, ZODB
try to implement conflict resolution yourself. In the future, ZODB
...
@@ -338,12 +337,12 @@ be freed if they aren't used later in the transaction.
...
@@ -338,12 +337,12 @@ be freed if they aren't used later in the transaction.
Concurrency, threads and processes
Concurrency, threads and processes
==================================
==================================
ZODB supports concurrency through transactions. Multiple programs
can
ZODB supports concurrency through transactions. Multiple programs
operate independently in separate transactions. They synchronize at
[#wtf-program]_ can operate independently in separate transactions.
transaction boundaries.
They synchronize at
transaction boundaries.
The most common way to run ZODB is with each program running in it's
The most common way to run ZODB is with each program running in it's
own thread. Usually
using
the thread-local transaction manager is used.
own thread. Usually the thread-local transaction manager is used.
You can use multiple threads per transaction and you can run multiple
You can use multiple threads per transaction and you can run multiple
transactions in a single thread. To do this, you need to instantiate
transactions in a single thread. To do this, you need to instantiate
...
@@ -356,7 +355,7 @@ To spread a transaction over multiple threads, you need to keep in
...
@@ -356,7 +355,7 @@ To spread a transaction over multiple threads, you need to keep in
mind that database connections, transaction managers and transactions
mind that database connections, transaction managers and transactions
are **not thread-safe**. You have to prevent simultaneous access from
are **not thread-safe**. You have to prevent simultaneous access from
multiple threads. For this reason, **using multiple threads with a
multiple threads. For this reason, **using multiple threads with a
single
conne
ction is not recommended**, but it is possible with care.
single
transa
ction is not recommended**, but it is possible with care.
Using multiple processes
Using multiple processes
------------------------
------------------------
...
@@ -385,10 +384,18 @@ Some things to keep in mind when utilizing multiple processes:
...
@@ -385,10 +384,18 @@ Some things to keep in mind when utilizing multiple processes:
in some cases, especially of subsequent transactions
in some cases, especially of subsequent transactions
haven't modified the same objects.
haven't modified the same objects.
.. [#bad-idea-using-multiple-threads-per-transaction] While it's
possible to spread transaction work over multiple threads, **it's
not a good idea**. See `Concurrency, threads and processes`_
.. [#implicit-transaction-creation] Transactions are implicitly
.. [#implicit-transaction-creation] Transactions are implicitly
created when needed, such as when data are first modified.
created when needed, such as when data are first modified.
.. [#context-managers-are-new] ZODB and the transaction package
.. [#context-managers-are-new] ZODB and the transaction package
predate context managers and the Python ``with`` statement.
predate context managers and the Python ``with`` statement.
.. [#wtf-program] We're using *program* here in a fairly general
sense, meaning some logic that we want to run to
perform some function, as opposed to an operating system program.
.. [#cant-share-now] at least not now.
.. [#cant-share-now] at least not now.
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment