Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
N
neoppod
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Analytics
Analytics
CI / CD
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
Ivan Tyagov
neoppod
Commits
d53cf5d2
Commit
d53cf5d2
authored
Sep 09, 2011
by
Julien Muchembled
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Some documentation updates and spelling fixes
parent
c56ffe39
Changes
4
Hide whitespace changes
Inline
Side-by-side
Showing
4 changed files
with
36 additions
and
22 deletions
+36
-22
README
README
+25
-14
TODO
TODO
+4
-6
neo/master/transactions.py
neo/master/transactions.py
+1
-1
setup.py
setup.py
+6
-1
No files found.
README
View file @
d53cf5d2
...
@@ -46,12 +46,15 @@ Requirements
...
@@ -46,12 +46,15 @@ Requirements
- For python 2.4: ctypes http://python.net/crew/theller/ctypes/
- For python 2.4: ctypes http://python.net/crew/theller/ctypes/
(packaged with later python versions)
(packaged with later python versions)
Note that setup.py does not define any dependency to 'ctypes' so you will
Note that setup.py does not define any dependency to 'ctypes' so you will
have to install it explicitely.
have to install it explicitely.
- MySQLdb http://sourceforge.net/projects/mysql-python
- For storage nodes:
- MySQLdb: http://sourceforge.net/projects/mysql-python
-
ZODB 3.10 or later (requires python 2.5 or later, only for client nodes)
-
For client nodes: ZODB 3.10.x but it should work with ZODB >= 3.4
Installation
Installation
============
============
...
@@ -74,12 +77,12 @@ d. Tell the cluster it can provide service::
...
@@ -74,12 +77,12 @@ d. Tell the cluster it can provide service::
How to use
How to use
==========
==========
First make sure Python can import 'neo.client' package.
In zope
In zope
-------
-------
a. Copy neo directory to /path/to/your/zope/lib/python
a. Edit your zope.conf, add a neo import and edit the `zodb_db` section by
b. Edit your zope.conf, add a neo import and edit the `zodb_db` section by
replacing its filestorage subsection by a NEOStorage one.
replacing its filestorage subsection by a NEOStorage one.
It should look like::
It should look like::
...
@@ -93,23 +96,31 @@ b. Edit your zope.conf, add a neo import and edit the `zodb_db` section by
...
@@ -93,23 +96,31 @@ b. Edit your zope.conf, add a neo import and edit the `zodb_db` section by
mount-point /
mount-point /
</zodb_db>
</zodb_db>
c
. Start zope
b
. Start zope
In a Python script
In a Python script
------------------
------------------
a. Set your PYTHONPATH in order to be able to import Zope lib::
Just create the storage object and play with it::
from neo.client.Storage import Storage
s = Storage(master_nodes="127.0.0.1:10010", name="main")
...
$ export PYTHONPATH=/path/to/your/zope/lib/python:$PYTHONPATH
"name" and "master_nodes" parameters have the same meaning as in
configuration file.
b. Just create the storage object and play with it::
Shutting down
-------------
from neo.client.Storage import Storage
There no administration command yet to stop properly a running cluster.
s = Storage(master_nodes="127.0.0.1:10010", name="main")
So following manual actions should be done:
...
"name" and "master_nodes" parameters have the same meaning as in
a. Make sure all clients like Zope instances are stopped, so that cluster
configuration file.
become idle.
b. Stop all master nodes first with a SIGINT or SIGTERM, so that storage nodes
don't become in OUT_OF_DATE state.
c. At last stop remaining nodes with a SIGINT or SIGTERM.
Deployment
Deployment
==========
==========
...
...
TODO
View file @
d53cf5d2
...
@@ -69,7 +69,7 @@ RC - Review output of pylint (CODE)
...
@@ -69,7 +69,7 @@ RC - Review output of pylint (CODE)
many places.
many places.
- Clarify handler methods to call when a connection is accepted from a
- Clarify handler methods to call when a connection is accepted from a
listening conenction and when remote node is identified
listening conenction and when remote node is identified
(cf. neo/bootstrap.py).
(cf. neo/
lib/
bootstrap.py).
- Choose how to handle a storage integrity verification when it comes back.
- Choose how to handle a storage integrity verification when it comes back.
Do the replication process, the verification stage, with or without
Do the replication process, the verification stage, with or without
unfinished transactions, cells have to set as outdated, if yes, should the
unfinished transactions, cells have to set as outdated, if yes, should the
...
@@ -77,7 +77,7 @@ RC - Review output of pylint (CODE)
...
@@ -77,7 +77,7 @@ RC - Review output of pylint (CODE)
- Implement proper shutdown (ClusterStates.STOPPING)
- Implement proper shutdown (ClusterStates.STOPPING)
- Review PENDING/HIDDEN/SHUTDOWN states, don't use notifyNodeInformation()
- Review PENDING/HIDDEN/SHUTDOWN states, don't use notifyNodeInformation()
to do a state-switch, use a exception-based mechanism ? (CODE)
to do a state-switch, use a exception-based mechanism ? (CODE)
- Split protocol.py in a 'protocol' module
- Split protocol.py in a 'protocol' module
?
- Review handler split (CODE)
- Review handler split (CODE)
The current handler split is the result of small incremental changes. A
The current handler split is the result of small incremental changes. A
global review is required to make them square.
global review is required to make them square.
...
@@ -98,7 +98,7 @@ RC - Review output of pylint (CODE)
...
@@ -98,7 +98,7 @@ RC - Review output of pylint (CODE)
committed by future transactions.
committed by future transactions.
Storage
Storage
- Use
Kyoto Cabinet
instead of a stand-alone MySQL server.
- Use
HailDB
instead of a stand-alone MySQL server.
- Notify master when storage becomes available for clients (LATENCY)
- Notify master when storage becomes available for clients (LATENCY)
Currently, storage presence is broadcasted to client nodes too early, as
Currently, storage presence is broadcasted to client nodes too early, as
the storage node would refuse them until it has only up-to-date data (not
the storage node would refuse them until it has only up-to-date data (not
...
@@ -163,7 +163,7 @@ RC - Review output of pylint (CODE)
...
@@ -163,7 +163,7 @@ RC - Review output of pylint (CODE)
- Master node data redundancy (HIGH AVAILABILITY)
- Master node data redundancy (HIGH AVAILABILITY)
Secondary master nodes should replicate primary master data (ie, primary
Secondary master nodes should replicate primary master data (ie, primary
master should inform them of such changes).
master should inform them of such changes).
This data takes too long to extract from storage nodes, and lo
o
sing it
This data takes too long to extract from storage nodes, and losing it
increases the risk of starting from underestimated values.
increases the risk of starting from underestimated values.
This risk is (currently) unavoidable when all nodes stop running, but this
This risk is (currently) unavoidable when all nodes stop running, but this
case must be avoided.
case must be avoided.
...
@@ -223,8 +223,6 @@ RC - Review output of pylint (CODE)
...
@@ -223,8 +223,6 @@ RC - Review output of pylint (CODE)
distributed lock mechanisms, ...)
distributed lock mechanisms, ...)
- Choose how to compute the storage size
- Choose how to compute the storage size
- Make storage check if the OID match with it's partitions during a store
- Make storage check if the OID match with it's partitions during a store
- Consider using out-of-band TCP feature.
- IPv6 support (address field, bind, name resolution)
- Investigate delta compression for stored data
- Investigate delta compression for stored data
Idea would be to have a few most recent revisions being stored fully, and
Idea would be to have a few most recent revisions being stored fully, and
older revision delta-compressed, in order to save space.
older revision delta-compressed, in order to save space.
...
...
neo/master/transactions.py
View file @
d53cf5d2
...
@@ -191,7 +191,7 @@ class Transaction(object):
...
@@ -191,7 +191,7 @@ class Transaction(object):
for it.
for it.
Does nothing if the node was not part of the transaction.
Does nothing if the node was not part of the transaction.
"""
"""
# XXX: We might lo
o
se information that a storage successfully locked
# XXX: We might lose information that a storage successfully locked
# data but was later found to be disconnected. This loss has no impact
# data but was later found to be disconnected. This loss has no impact
# on current code, but it might be disturbing to reader or future code.
# on current code, but it might be disturbing to reader or future code.
if
self
.
_prepared
:
if
self
.
_prepared
:
...
...
setup.py
View file @
d53cf5d2
"""Distributed, redundant and transactional storage for ZODB
"""
from
setuptools
import
setup
,
find_packages
from
setuptools
import
setup
,
find_packages
import
os
import
os
...
@@ -34,12 +37,14 @@ extras_require['tests'] = ['zope.testing', 'psutil',
...
@@ -34,12 +37,14 @@ extras_require['tests'] = ['zope.testing', 'psutil',
setup
(
setup
(
name
=
'neoppod'
,
name
=
'neoppod'
,
version
=
'1.0a1'
,
version
=
'1.0a1'
,
description
=
'Distributed, redundant and transactional storage for ZODB'
,
description
=
__doc__
.
strip
()
,
author
=
'NEOPPOD'
,
author
=
'NEOPPOD'
,
author_email
=
'neo-dev@erp5.org'
,
author_email
=
'neo-dev@erp5.org'
,
url
=
'http://www.neoppod.org/'
,
url
=
'http://www.neoppod.org/'
,
license
=
'GPL 2+'
,
license
=
'GPL 2+'
,
platforms
=
[
"any"
],
classifiers
=
classifiers
.
splitlines
(),
classifiers
=
classifiers
.
splitlines
(),
long_description
=
".. contents::
\
n
\
n
"
+
open
(
'README'
).
read
(),
packages
=
find_packages
(),
packages
=
find_packages
(),
py_modules
=
[
'mock'
],
py_modules
=
[
'mock'
],
entry_points
=
{
entry_points
=
{
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment