Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
N
neo
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Analytics
Analytics
CI / CD
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
Stefane Fermigier
neo
Commits
224433cd
Commit
224433cd
authored
Aug 23, 2012
by
Julien Muchembled
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Update documentation
parent
2c170443
Changes
4
Hide whitespace changes
Inline
Side-by-side
Showing
4 changed files
with
95 additions
and
26 deletions
+95
-26
BUGS
BUGS
+30
-0
README
README
+46
-15
neo.conf
neo.conf
+18
-10
neo/neoctl/app.py
neo/neoctl/app.py
+1
-1
No files found.
BUGS
0 → 100644
View file @
224433cd
Although NEO is considered ready for production use in most cases, there are
a few bugs to know because they concern basic features of ZODB, or promised
features of NEO.
All the listed bugs will be fixed with high priority.
Conflict resolution not fully implemented
-----------------------------------------
When a cluster has several storage nodes, so-called 'deadlock avoidance' may
happen to in order to resolve conflicts. In such cases, conflicts will not be
resolved even if your _p_resolveConflict() method would succeed, leading to a
normal ConflictError.
Although this should happen rarely enough not to affect performance, this can
be an issue if your application can't afford restarting the transaction,
e.g. because it interacted with external environment.
Master failure not recovered by clients
---------------------------------------
After a master failure, clients are stuck in a bad state. In particular, caches
are not flushed, which could lead to data corruption if they reconnect a master.
This makes secondary masters mostly useless for the moment.
Storage failure or update may lead to POSException or break undoLog()
---------------------------------------------------------------------
Storage nodes are only queried once at most and if all (for the requested
partition) failed, the client raises instead of asking the master whether it
had an up-to-date partition table (and retry if useful).
In the case of undoLog(), incomplete results may be returned.
README
View file @
224433cd
...
...
@@ -14,9 +14,9 @@ A NEO cluster is composed of the following types of nodes:
- "storage" nodes (mandatory, 1 or more)
Stores data
in a MySQL database, preserving history. All available storag
e
nodes are in use
simultaneously. This offers redundancy and data distribution.
Other storage backends than MySQL are considered for future release.
Stores data
, preserving history. All available storage nodes are in us
e
simultaneously. This offers redundancy and data distribution.
Available backends: MySQL, SQLite
- "admin" nodes (mandatory for startup, optional after)
...
...
@@ -56,29 +56,43 @@ Requirements
- Python 2.6.x or 2.7.x
- For storage nodes:
- For storage nodes
using MySQL backend
:
- MySQLdb: http://sourceforge.net/projects/mysql-python
- For client nodes: ZODB 3.
10.x but it should work with ZODB 3.9.x
- For client nodes: ZODB 3.
9.x or later
Installation
============
a. Make neo directory available for python to import (for example, by
a. NEO can be installed like any other egg (see setup.py). Or you can simply
make `neo` directory available for Python to import (for example, by
adding its container directory to the PYTHONPATH environment variable).
b. Choose a cluster name and setup a MySQL database
b. Write a neo.conf file like the example provided. If you use MySQL,
you'll also need create 1 database per storage node.
c. Start all required nodes::
neomaster --cluster=<cluster name>
neostorage --cluster=<cluster name> --database=user:passwd@db
neoadmin --cluster=<cluster name>
$ neomaster -f neo.conf
$ neostorage -f neo.conf -s storage1
$ neostorage -f neo.conf -s storage2
$ neoadmin -f neo.conf
d. Tell the cluster
it can provide service
::
d. Tell the cluster
to initialize storage nodes
::
neoctl -a <admin> start
$ neoctl -a <admin> start
e. Clients can connect when the cluster is in RUNNING state::
$ neoctl -a <admin> print cluster
RUNNING
f. See `neomigrate` command to import an existing FileStorage database,
or `neoctl` command for more administrative tasks.
Alternatively, you can use `neosimple` command to quickly setup a cluster for
testing.
How to use
==========
...
...
@@ -134,6 +148,23 @@ This can also be done manually, which helps if your cluster is in bad state:
don't become in OUT_OF_DATE state.
- Next stop remaining nodes with a SIGINT or SIGTERM.
Master-slave asynchronous replication
-------------------------------------
This is the recommanded way to backup a NEO cluster.
Once a cluster with appropriate `upstream_cluster` & `upstream_masters`
configuration is started, you can switch it into backup mode
using::
neoctl -a <admin> set cluster STARTING_BACKUP
It remembers it is in such mode when it is stopped, and it can be put back into
normal mode (RUNNING) by setting it into STOPPING_BACKUP state.
Packs are currently not replicated, which means packing should always be done
up to a TID that is already fully replicated, so that the backup cluster has a
full history (and not random holes).
Deployment
==========
...
...
@@ -145,15 +176,15 @@ NEO has no built-in deployment features such as process daemonization. We use
[program:storage_01]
priority=10
command=neostorage -
c neo -
s storage_01 -f /neo/neo.conf
command=neostorage -s storage_01 -f /neo/neo.conf
[program:master_01]
priority=20
command=neomaster -
c neo -
s master_01 -f /neo/neo.conf
command=neomaster -s master_01 -f /neo/neo.conf
[program:admin]
priority=20
command=neoadmin -
c neo -
s admin -f /neo/neo.conf
command=neoadmin -s admin -f /neo/neo.conf
Developers
==========
...
...
neo.conf
View file @
224433cd
# Note: Unless otherwise noted, all parameters in this configuration file
# must be identical for all nodes in a given cluster.
# This file is optionnal: parameters can be given at the command line.
#
Default
parameters.
#
Common
parameters.
[
DEFAULT
]
# The cluster name
# This must be set.
# It must be a name unique to a given cluster, to prevent foreign
# I
MPORTANT: I
t must be a name unique to a given cluster, to prevent foreign
# misconfigured nodes from interfering.
cluster
:
cluster
:
test
# The list of master nodes
# Master nodes not not in this list will be rejected by the cluster.
...
...
@@ -22,18 +23,25 @@ masters: 127.0.0.1:10000
# 0 means no redundancy
# 1 means there is a spare copy of all partitions
replicas
:
1
# Partitions: How data spreads among storage nodes. This number must be at
# least equal to the number of storage nodes the cluster contains.
# IMPORTANT: This must not be changed once the cluster contains data.
partitions
:
20
# Partitions: How data spreads among storage nodes.
# IMPORTANT: This can not be changed once the cluster contains data.
partitions
:
12
# The maximum number of usable storage nodes is: partitions * (replicas + 1)
# Master-slave asynchronous replication
# The following 2 options are only required if you want to backup another
# NEO cluster.
;
upstream_cluster
:
test2
;
upstream_masters
:
127
.
0
.
0
.
1
:
30000
# Individual nodes parameters
# Some parameters makes no sense to be defined in [DEFAULT] section.
# They are:
# bind: The ip:port the node will listen on.
# database: Storage nodes only.
# Syntax for MySQL database is [user[:password]@]database[unix_socket]
# Those database must be created manualy.
# database: Storage nodes only. Syntax for:
# - MySQL: [user[:password]@]database[unix_socket]
# Database must be created manually.
# - SQLite: path
# Admin node
[
admin
]
...
...
neo/neoctl/app.py
View file @
224433cd
...
...
@@ -195,7 +195,7 @@ class TerminalNeoCTL(object):
def
checkReplicas
(
self
,
params
):
"""
Test whether partitions have corrupted meta
Test whether partitions have corrupted meta
data
Any corrupted cell is put in CORRUPTED state, possibly make the
cluster non operational.
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment