Skip to content

Z
ZEO
Commit 5c2fe859 authored by Jim Fulton's avatar Jim Fulton
Browse files
Options

Merged changes from ZEO-Dev branch in preparation for 0.2

parent 5f6e7d61
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 1355 additions and 429 deletions
src/ZEO/CHANGES.txt 0 → 100644
View file @ 5c2fe859
Zope Enterprise Option (ZEO) Revision History
ZEO 0.2
This release is expected to be close to beta quality. Initially, the
primary goals of this release were to:
- Correct some consistency problems that had been observed in
0.1 on starup.
- Allow ZEO clients to detect, survive, and recover from
disconnection from the ZEO server.
Based on some feedback from some folks who tried 0.1, improving
write performance was made a priority.
Features
- The ZEO Client now handles server failures gracefully:
o The client with a persistent cache can generally startup
even if the server is not running, assuming that it has at
least a minimal number of objects in the cache.
o The client will continue to function even if the server
connection is interuppted.
o Server availability is detected by the client (which tries
to connect to the server every few minutes). A disconnected
client will automatically reconnect to an available server.
o When the client is disconnected, write transactions cannot
be performed. Reads fail for objects that are not in the
cache.
- Performance enhancements
The speed of write-intensive operations have been improved
approximately 70%. When using Unix domain sockets for
client/server communication, ZEO transactions take roughly 2-3
times as long as FileStorage transactions to commit.
(This was based on some tests. Your mileage may vary.)
- Packing support was added. Note that packing is done
asynchrounously. The client returns immediately from a pack
call. The server packs in a thread and sends updated
statistics to the client when packing is completed.
- Support for Unix-domain sockets was added.
- Pickles sent to the server are now checked to make sure that
they don't contain unapproved instance or global-variable
(function) pickles.
Bugs fixed
- Data could be badly inconsistent when a persistent cache
was started, due to a bug in the cache initialization logic.
- The application was allowed to begin operation while the cache
was being verified. This could lead to harmful inconsistencies.
Changes made to Zope to support ZEO
- A number of changes were made to ZODB to support asynchronous
storage during transaction commit.
- Normally Zope updates the database during startup to reflect
product changes. This behavior is now suppressed when the
ZEO_CLIENT environment variable is set. It doesn't make sense
for many clients to update the database for the same products.
- The asyncore module was modified to add support for multiple
asyncore loops. This change was applied to asyncore in the
Zope and the (official, owned by Sam Rushing) medusa CVS
trees.
- A new module, ThreadedAsync.py has been added in the Zope
lib/python directory. This module provides notification to
async objects (like ZEO clients) to let them know when the
asyncore main loop has started. This was needed to enable use
of async code before the main loop starts.
ZEO 0.1 (aka "iteration 1")
This was an initial alpha of ZEO that demonstrated basic
functionalities. It lacked robustness and has some performance
problems on writes.
src/ZEO/ClientCache.py
View file @ 5c2fe859
###################################################################### ##############################################################################
# Digital Creations Options License Version 0.9.0
# -----------------------------------------------
# #
# Copyright (c) 1999, Digital Creations. All rights reserved. # Zope Public License (ZPL) Version 1.0
# -------------------------------------
# #
# This license covers Zope software delivered as "options" by Digital # Copyright (c) Digital Creations. All rights reserved.
# Creations.
# #
# Use in source and binary forms, with or without modification, are # This license has been certified as Open Source(tm).
# permitted provided that the following conditions are met:
# #
# 1. Redistributions are not permitted in any form. # Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
# #
# 2. This license permits one copy of software to be used by up to five # 1. Redistributions in source code must retain the above copyright
# developers in a single company. Use by more than five developers # notice, this list of conditions, and the following disclaimer.
# requires additional licenses.
# #
# 3. Software may be used to operate any type of website, including # 2. Redistributions in binary form must reproduce the above copyright
# publicly accessible ones. # notice, this list of conditions, and the following disclaimer in
# the documentation and/or other materials provided with the
# distribution.
# #
# 4. Software is not fully documented, and the customer acknowledges # 3. Digital Creations requests that attribution be given to Zope
# that the product can best be utilized by reading the source code. # in any manner possible. Zope includes a "Powered by Zope"
# button that is installed by default. While it is not a license
# violation to remove this button, it is requested that the
# attribution remain. A significant investment has been put
# into Zope, and this effort will continue if the Zope community
# continues to grow. This is one way to assure that growth.
#
# 4. All advertising materials and documentation mentioning
# features derived from or use of this software must display
# the following acknowledgement:
#
# "This product includes software developed by Digital Creations
# for use in the Z Object Publishing Environment
# (http://www.zope.org/)."
#
# In the event that the product being advertised includes an
# intact Zope distribution (with copyright and license included)
# then this clause is waived.
#
# 5. Names associated with Zope or Digital Creations must not be used to
# endorse or promote products derived from this software without
# prior written permission from Digital Creations.
#
# 6. Modified redistributions of any form whatsoever must retain
# the following acknowledgment:
#
# "This product includes software developed by Digital Creations
# for use in the Z Object Publishing Environment
# (http://www.zope.org/)."
#
# Intact (re-)distributions of any official Zope release do not
# require an external acknowledgement.
#
# 7. Modifications are encouraged but must be packaged separately as
# patches to official Zope releases. Distributions that do not
# clearly separate the patches from the original work must be clearly
# labeled as unofficial distributions. Modifications which do not
# carry the name Zope may be packaged in any form, as long as they
# conform to all of the clauses above.
# #
# 5. Support for software is included for 90 days in email only. Further
# support can be purchased separately.
# #
# Disclaimer # Disclaimer
# #
...@@ -39,7 +75,13 @@ ...@@ -39,7 +75,13 @@
# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT # OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
# SUCH DAMAGE. # SUCH DAMAGE.
###################################################################### #
#
# This software consists of contributions made by Digital Creations and
# many individuals on behalf of Digital Creations. Specific
# attributions are listed in the accompanying credits file.
#
##############################################################################
"""Implement a client cache """Implement a client cache
The cache is managed as two files, var/c0.zec and var/c1.zec. The cache is managed as two files, var/c0.zec and var/c1.zec.
...@@ -102,7 +144,7 @@ file 0 and file 1. ...@@ -102,7 +144,7 @@ file 0 and file 1.
""" """
__version__ = "$Revision: 1.4 $"[11:-2] __version__ = "$Revision: 1.5 $"[11:-2]
import os, tempfile import os, tempfile
from struct import pack, unpack from struct import pack, unpack
...@@ -111,9 +153,9 @@ magic='ZEC0' ...@@ -111,9 +153,9 @@ magic='ZEC0'
class ClientCache: class ClientCache:
def __init__(self, storage='', size=20000000, client=None): def __init__(self, storage='', size=20000000, client=None, var=None):
var=os.path.join(INSTANCE_HOME,'var')
if client: if client:
if var is None: var=os.path.join(INSTANCE_HOME,'var')
self._p=p=map(lambda i, p=storage, var=var, c=client: self._p=p=map(lambda i, p=storage, var=var, c=client:
os.path.join(var,'c%s-%s-%s.zec' % (p, c, i)), os.path.join(var,'c%s-%s-%s.zec' % (p, c, i)),
(0,1)) (0,1))
...@@ -317,6 +359,12 @@ def read_index(index, serial, f, current): ...@@ -317,6 +359,12 @@ def read_index(index, serial, f, current):
if current: index[oid]=-pos if current: index[oid]=-pos
else: index[oid]=pos else: index[oid]=pos
serial[oid]=h[-8:], vs serial[oid]=h[-8:], vs
else:
if serial.has_key(oid):
# We has a record for this oid, but it was invalidated!
del serial[oid]
del index[oid]
pos=pos+tlen pos=pos+tlen
......
src/ZEO/ClientStorage.py
View file @ 5c2fe859
This diff is collapsed.
src/ZEO/Invalidator.py
View file @ 5c2fe859
##############################################################################
#
# Zope Public License (ZPL) Version 1.0
# -------------------------------------
#
# Copyright (c) Digital Creations. All rights reserved.
#
# This license has been certified as Open Source(tm).
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# 1. Redistributions in source code must retain the above copyright
# notice, this list of conditions, and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions, and the following disclaimer in
# the documentation and/or other materials provided with the
# distribution.
#
# 3. Digital Creations requests that attribution be given to Zope
# in any manner possible. Zope includes a "Powered by Zope"
# button that is installed by default. While it is not a license
# violation to remove this button, it is requested that the
# attribution remain. A significant investment has been put
# into Zope, and this effort will continue if the Zope community
# continues to grow. This is one way to assure that growth.
#
# 4. All advertising materials and documentation mentioning
# features derived from or use of this software must display
# the following acknowledgement:
#
# "This product includes software developed by Digital Creations
# for use in the Z Object Publishing Environment
# (http://www.zope.org/)."
#
# In the event that the product being advertised includes an
# intact Zope distribution (with copyright and license included)
# then this clause is waived.
#
# 5. Names associated with Zope or Digital Creations must not be used to
# endorse or promote products derived from this software without
# prior written permission from Digital Creations.
#
# 6. Modified redistributions of any form whatsoever must retain
# the following acknowledgment:
#
# "This product includes software developed by Digital Creations
# for use in the Z Object Publishing Environment
# (http://www.zope.org/)."
#
# Intact (re-)distributions of any official Zope release do not
# require an external acknowledgement.
#
# 7. Modifications are encouraged but must be packaged separately as
# patches to official Zope releases. Distributions that do not
# clearly separate the patches from the original work must be clearly
# labeled as unofficial distributions. Modifications which do not
# carry the name Zope may be packaged in any form, as long as they
# conform to all of the clauses above.
#
#
# Disclaimer
#
# THIS SOFTWARE IS PROVIDED BY DIGITAL CREATIONS ``AS IS'' AND ANY
# EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL DIGITAL CREATIONS OR ITS
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
# SUCH DAMAGE.
#
#
# This software consists of contributions made by Digital Creations and
# many individuals on behalf of Digital Creations. Specific
# attributions are listed in the accompanying credits file.
#
##############################################################################
"""Facility for (roughly) atomically invalidating cache entries. """Facility for (roughly) atomically invalidating cache entries.
Note that this is not *really* atomic, but it is close enough. Note that this is not *really* atomic, but it is close enough.
...@@ -15,16 +99,17 @@ class Invalidator: ...@@ -15,16 +99,17 @@ class Invalidator:
def begin(self): def begin(self):
self._tfile=tempfile.TemporaryFile() self._tfile=tempfile.TemporaryFile()
self._d=cPickle.Pickler(self._tfile).dump pickler=cPickle.Pickler(self._tfile, 1)
pickler.fast=1 # Don't use the memo
self._d=pickler.dump
def invalidate(self, args): def invalidate(self, args):
if self._d is None: return if self._d is None: return
for arg in args: self._d(args)
self._d(arg)
def end(self): def end(self):
if self._d is None: return if self._d is None: return
self._d((0,0,0)) self._d((0,0))
self._d=None self._d=None
self._tfile.seek(0) self._tfile.seek(0)
load=cPickle.Unpickler(self._tfile).load load=cPickle.Unpickler(self._tfile).load
...@@ -34,8 +119,15 @@ class Invalidator: ...@@ -34,8 +119,15 @@ class Invalidator:
dinvalidate=self.dinvalidate dinvalidate=self.dinvalidate
while 1: while 1:
oid, serial, version = load() oid, version = load()
if not oid: break if not oid: break
cinvalidate(oid, version=version) cinvalidate(oid, version=version)
dinvalidate(oid, version=version) dinvalidate(oid, version=version)
def Invalidate(self, args):
if self._d is None: return
cinvalidate=self.cinvalidate
dinvalidate=self.dinvalidate
for oid, version in args:
cinvalidate(oid, version=version)
dinvalidate(oid, version=version)
src/ZEO/README
View file @ 5c2fe859
Zope Enterprize Option, iteration 1 Zope Enterprize Option, ZEO 0.2
Put this package in your Zope lib/python. Put this package in your Zope lib/python.
Note -- This release of ZEO requires Zope 2.2 or a CVS checkout
of Zope. See 'CHANGES.txt' for details.
You also need to symbolically link (or copy) ZServer to lib/python::
cd lib/python
ln -s ../../ZServer .
To start the storage server, go to your Zope install directory and:: To start the storage server, go to your Zope install directory and::
lib/python/ZEO/start.py -p port_number lib/python/ZEO/start.py -p port_number
(Run start without arguments to see options.) (Run start without arguments to see options.)
Of course, the server and the client don't have to be on the same
machine.
If the server and client *are* on the same machine, then you can use
a Unix domain socket::
lib/python/ZEO/start.py -U filename
To get Zope to use the server, create a custom_zodb module, To get Zope to use the server, create a custom_zodb module,
custom_zodb.py, in your Zope install directory, so that uses a custom_zodb.py, in your Zope install directory, so that Zope uses a
ClientStorage:: ClientStorage::
import ZEO.ClientStorage import ZEO.ClientStorage
Storage=ZEO.ClientStorage.ClientStorage(('',port_number), async=1) Storage=ZEO.ClientStorage.ClientStorage(('',port_number))
You can specify a host name (rather than '') if you want. You can specify a host name (rather than '') if you want.
The port number is, of course, the port number used to start the The port number is, of course, the port number used to start the
storage server. The async switch tells the client to switch storage server. The async switch tells the client to switch
itself to async mode (if and) when the asyncore main loop is called. itself to async mode (if and) when the asyncore main loop is called.
You can also give the name of a Unix domain socket file::
import ZEO.ClientStorage
Storage=ZEO.ClientStorage.ClientStorage(filename)
If you want a persistent client cache, you need to define the If you want a persistent client cache, you need to define the
environment variable, ZEO_CLIENT to a unique name for the environment variable, ZEO_CLIENT to a unique name for the
client. This is needed so that unique cache name files can be client. This is needed so that unique cache name files can be
...@@ -28,4 +50,46 @@ Zope Enterprize Option, iteration 1 ...@@ -28,4 +50,46 @@ Zope Enterprize Option, iteration 1
something like: something like:
python z2.py -P8700 ZEO_CLIENT=8700 python z2.py -P8700 ZEO_CLIENT=8700
python z2.py -P8800 ZEO_CLIENT=8700 python z2.py -P8800 ZEO_CLIENT=8800
The ClientStorage constructor provides a number of additional
options (arguments):
storage -- The name of the storage to connect to.
cache_size -- The number of bytes to allow for the client cache.
The default is 20,000,000.
name -- The name to use for the storage. This will be shown in
Zope's control panel. The default name is a representation of
the connection information.
debug -- If this is provided, it should be a non-empty string. It
indicates that client should log tracing and debugging
information, using zLOG.
var -- The directory in which persistent cache files should be
written. If this option is provided, it is unnecessary to
set INSTANCE_HOME in __builtins__.
Notes for non Zope users
First, we regret the amount of dependence on Zope. We intend for
the dependence to decrease in the long run.
Known dependencies:
- Shared must be in the Python path. This is due to a lame
dependency on some Zope XML facilities used by ZODB for XML
export and import.
- ZServer should be in the Python path, or you should copy the
version of asyncore.py from ZServer (from Zope 2.2 or CVS) to
your Python path, or you should copy a version of a asyncore
from the medusa CVS tree to your Python path. A recent change
in asyncore is required.
- The module, ThreadedAsync must be in the python path.
src/ZEO/StorageServer.py
View file @ 5c2fe859
This diff is collapsed.
src/ZEO/__init__.py
View file @ 5c2fe859
""" Soon to be the ZPL """ ##############################################################################
#
# Zope Public License (ZPL) Version 1.0
# -------------------------------------
#
# Copyright (c) Digital Creations. All rights reserved.
#
# This license has been certified as Open Source(tm).
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# 1. Redistributions in source code must retain the above copyright
# notice, this list of conditions, and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions, and the following disclaimer in
# the documentation and/or other materials provided with the
# distribution.
#
# 3. Digital Creations requests that attribution be given to Zope
# in any manner possible. Zope includes a "Powered by Zope"
# button that is installed by default. While it is not a license
# violation to remove this button, it is requested that the
# attribution remain. A significant investment has been put
# into Zope, and this effort will continue if the Zope community
# continues to grow. This is one way to assure that growth.
#
# 4. All advertising materials and documentation mentioning
# features derived from or use of this software must display
# the following acknowledgement:
#
# "This product includes software developed by Digital Creations
# for use in the Z Object Publishing Environment
# (http://www.zope.org/)."
#
# In the event that the product being advertised includes an
# intact Zope distribution (with copyright and license included)
# then this clause is waived.
#
# 5. Names associated with Zope or Digital Creations must not be used to
# endorse or promote products derived from this software without
# prior written permission from Digital Creations.
#
# 6. Modified redistributions of any form whatsoever must retain
# the following acknowledgment:
#
# "This product includes software developed by Digital Creations
# for use in the Z Object Publishing Environment
# (http://www.zope.org/)."
#
# Intact (re-)distributions of any official Zope release do not
# require an external acknowledgement.
#
# 7. Modifications are encouraged but must be packaged separately as
# patches to official Zope releases. Distributions that do not
# clearly separate the patches from the original work must be clearly
# labeled as unofficial distributions. Modifications which do not
# carry the name Zope may be packaged in any form, as long as they
# conform to all of the clauses above.
#
#
# Disclaimer
#
# THIS SOFTWARE IS PROVIDED BY DIGITAL CREATIONS ``AS IS'' AND ANY
# EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL DIGITAL CREATIONS OR ITS
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
# SUCH DAMAGE.
#
#
# This software consists of contributions made by Digital Creations and
# many individuals on behalf of Digital Creations. Specific
# attributions are listed in the accompanying credits file.
#
##############################################################################
src/ZEO/custom_zodb.py.dist
View file @ 5c2fe859
...@@ -3,7 +3,7 @@ ...@@ -3,7 +3,7 @@
# This file tells Zope to look to a ZEO Storage Server for all of its # This file tells Zope to look to a ZEO Storage Server for all of its
# database objects. # database objects.
__version__ = "$Revision: 1.4 $"[11:-2] __version__ = "$Revision: 1.5 $"[11:-2]
# First we have to get the right ZEO components # First we have to get the right ZEO components
import ZEO.ClientStorage import ZEO.ClientStorage
...@@ -12,10 +12,9 @@ import ZEO.ClientStorage ...@@ -12,10 +12,9 @@ import ZEO.ClientStorage
ZSS_HOST = '' # Host name where ZSS is running ZSS_HOST = '' # Host name where ZSS is running
ZSS_PORT = 8800 # Port that the ZSS is running on ZSS_PORT = 8800 # Port that the ZSS is running on
ZSS_ASYNC = 1 # Set to 1 for async calls
ZSS_NAME = 'ZEOStorage' # Name of the storage being used ZSS_NAME = 'ZEOStorage' # Name of the storage being used
# Now we tell Zope where its storage is: # Now we tell Zope where its storage is:
Storage=ZEO.ClientStorage.ClientStorage((ZSS_HOST, ZSS_PORT), Storage=ZEO.ClientStorage.ClientStorage((ZSS_HOST, ZSS_PORT),
async=ZSS_ASYNC, name=ZSS_NAME) name=ZSS_NAME)
src/ZEO/smac.py
View file @ 5c2fe859
###################################################################### ##############################################################################
# Digital Creations Options License Version 0.9.0
# -----------------------------------------------
# #
# Copyright (c) 1999, Digital Creations. All rights reserved. # Zope Public License (ZPL) Version 1.0
# -------------------------------------
# #
# This license covers Zope software delivered as "options" by Digital # Copyright (c) Digital Creations. All rights reserved.
# Creations.
# #
# Use in source and binary forms, with or without modification, are # This license has been certified as Open Source(tm).
# permitted provided that the following conditions are met:
# #
# 1. Redistributions are not permitted in any form. # Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
# #
# 2. This license permits one copy of software to be used by up to five # 1. Redistributions in source code must retain the above copyright
# developers in a single company. Use by more than five developers # notice, this list of conditions, and the following disclaimer.
# requires additional licenses.
# #
# 3. Software may be used to operate any type of website, including # 2. Redistributions in binary form must reproduce the above copyright
# publicly accessible ones. # notice, this list of conditions, and the following disclaimer in
# the documentation and/or other materials provided with the
# distribution.
# #
# 4. Software is not fully documented, and the customer acknowledges # 3. Digital Creations requests that attribution be given to Zope
# that the product can best be utilized by reading the source code. # in any manner possible. Zope includes a "Powered by Zope"
# button that is installed by default. While it is not a license
# violation to remove this button, it is requested that the
# attribution remain. A significant investment has been put
# into Zope, and this effort will continue if the Zope community
# continues to grow. This is one way to assure that growth.
#
# 4. All advertising materials and documentation mentioning
# features derived from or use of this software must display
# the following acknowledgement:
#
# "This product includes software developed by Digital Creations
# for use in the Z Object Publishing Environment
# (http://www.zope.org/)."
#
# In the event that the product being advertised includes an
# intact Zope distribution (with copyright and license included)
# then this clause is waived.
#
# 5. Names associated with Zope or Digital Creations must not be used to
# endorse or promote products derived from this software without
# prior written permission from Digital Creations.
#
# 6. Modified redistributions of any form whatsoever must retain
# the following acknowledgment:
#
# "This product includes software developed by Digital Creations
# for use in the Z Object Publishing Environment
# (http://www.zope.org/)."
#
# Intact (re-)distributions of any official Zope release do not
# require an external acknowledgement.
#
# 7. Modifications are encouraged but must be packaged separately as
# patches to official Zope releases. Distributions that do not
# clearly separate the patches from the original work must be clearly
# labeled as unofficial distributions. Modifications which do not
# carry the name Zope may be packaged in any form, as long as they
# conform to all of the clauses above.
# #
# 5. Support for software is included for 90 days in email only. Further
# support can be purchased separately.
# #
# Disclaimer # Disclaimer
# #
...@@ -39,79 +75,118 @@ ...@@ -39,79 +75,118 @@
# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT # OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
# SUCH DAMAGE. # SUCH DAMAGE.
###################################################################### #
#
# This software consists of contributions made by Digital Creations and
# many individuals on behalf of Digital Creations. Specific
# attributions are listed in the accompanying credits file.
#
##############################################################################
"""Sized message async connections """Sized message async connections
""" """
__version__ = "$Revision: 1.6 $"[11:-2] __version__ = "$Revision: 1.7 $"[11:-2]
import asyncore, string, struct, zLOG, sys, Acquisition
from zLOG import LOG, TRACE, ERROR
import asyncore, string, struct, zLOG class SizedMessageAsyncConnection(Acquisition.Explicit, asyncore.dispatcher):
from zLOG import LOG, INFO, ERROR
class SizedMessageAsyncConnection(asyncore.dispatcher): __append=None # Marker indicating that we're closed
def __init__(self, sock, addr): socket=None # to outwit Sam's getattr
asyncore.dispatcher.__init__(self, sock)
def __init__(self, sock, addr, map=None, debug=None):
SizedMessageAsyncConnection.inheritedAttribute(
'__init__')(self, sock, map)
self.addr=addr self.addr=addr
if debug is not None:
self._debug=debug
elif not hasattr(self, '_debug'):
self._debug=__debug__ and 'smac'
self.__state=None self.__state=None
self.__inp=None self.__inp=None
self.__inpl=0
self.__l=4 self.__l=4
self.__output=output=[] self.__output=output=[]
self.__append=output.append self.__append=output.append
self.__pop=output.pop self.__pop=output.pop
def handle_read(self, def handle_read(self,
join=string.join, StringType=type('')): join=string.join, StringType=type(''), _type=type,
l=self.__l StringType=type(''), _None=None):
d=self.recv(l)
d=self.recv(8096)
if not d: return
inp=self.__inp inp=self.__inp
if inp is None: if inp is _None:
inp=d inp=d
elif type(inp) is StringType: elif _type(inp) is StringType:
inp=[inp,d] inp=[inp,d]
else: else:
inp.append(d) inp.append(d)
l=l-len(d) inpl=self.__inpl+len(d)
if l <= 0: l=self.__l
if type(inp) is not StringType: inp=join(inp,'')
if self.__state is None: while 1:
# waiting for message
self.__l=struct.unpack(">i",inp)[0] if l <= inpl:
self.__state=1 # Woo hoo, we have enough data
self.__inp=None if _type(inp) is not StringType: inp=join(inp,'')
d=inp[:l]
inp=inp[l:]
inpl=inpl-l
if self.__state is _None:
# waiting for message
l=struct.unpack(">i",d)[0]
self.__state=1
else:
l=4
self.__state=_None
self.message_input(d)
else: else:
self.__inp=None break # not enough data
self.__l=4
self.__state=None self.__l=l
self.message_input(inp) self.__inp=inp
else: self.__inpl=inpl
self.__l=l
self.__inp=inp
def readable(self): return 1 def readable(self): return 1
def writable(self): return not not self.__output def writable(self): return not not self.__output
def handle_write(self): def handle_write(self):
output=self.__output output=self.__output
if output: while output:
v=output[0] v=output[0]
n=self.send(v) n=self.send(v)
if n < len(v): if n < len(v):
output[0]=v[n:] output[0]=v[n:]
break # we can't write any more
else: else:
del output[0] del output[0]
#break # waaa
def handle_close(self): def handle_close(self):
self.close() self.close()
def message_output(self, message, def message_output(self, message,
pack=struct.pack, len=len): pack=struct.pack, len=len):
if __debug__: if self._debug:
if len(message) > 40: m=message[:40]+' ...' if len(message) > 40: m=message[:40]+' ...'
else: m=message else: m=message
LOG('smax', INFO, 'message_output %s' % `m`) LOG(self._debug, TRACE, 'message_output %s' % `m`)
self.__append(pack(">i",len(message))+message)
append=self.__append
if append is None:
raise Disconnected, (
"This action is temporarily unavailable."
"<p>"
)
append(pack(">i",len(message))+message)
def log_info(self, message, type='info'): def log_info(self, message, type='info'):
if type=='error': type=ERROR if type=='error': type=ERROR
...@@ -119,3 +194,13 @@ class SizedMessageAsyncConnection(asyncore.dispatcher): ...@@ -119,3 +194,13 @@ class SizedMessageAsyncConnection(asyncore.dispatcher):
LOG('ZEO', type, message) LOG('ZEO', type, message)
log=log_info log=log_info
def close(self):
if self.__append is not None:
self.__append=None
SizedMessageAsyncConnection.inheritedAttribute('close')(self)
class Disconnected(Exception):
"""The client has become disconnected from the server
"""
src/ZEO/start.py
View file @ 5c2fe859
###################################################################### ##############################################################################
# Digital Creations Options License Version 0.9.0
# -----------------------------------------------
# #
# Copyright (c) 1999, Digital Creations. All rights reserved. # Zope Public License (ZPL) Version 1.0
# -------------------------------------
# #
# This license covers Zope software delivered as "options" by Digital # Copyright (c) Digital Creations. All rights reserved.
# Creations.
# #
# Use in source and binary forms, with or without modification, are # This license has been certified as Open Source(tm).
# permitted provided that the following conditions are met:
# #
# 1. Redistributions are not permitted in any form. # Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
# #
# 2. This license permits one copy of software to be used by up to five # 1. Redistributions in source code must retain the above copyright
# developers in a single company. Use by more than five developers # notice, this list of conditions, and the following disclaimer.
# requires additional licenses.
# #
# 3. Software may be used to operate any type of website, including # 2. Redistributions in binary form must reproduce the above copyright
# publicly accessible ones. # notice, this list of conditions, and the following disclaimer in
# the documentation and/or other materials provided with the
# distribution.
# #
# 4. Software is not fully documented, and the customer acknowledges # 3. Digital Creations requests that attribution be given to Zope
# that the product can best be utilized by reading the source code. # in any manner possible. Zope includes a "Powered by Zope"
# button that is installed by default. While it is not a license
# violation to remove this button, it is requested that the
# attribution remain. A significant investment has been put
# into Zope, and this effort will continue if the Zope community
# continues to grow. This is one way to assure that growth.
#
# 4. All advertising materials and documentation mentioning
# features derived from or use of this software must display
# the following acknowledgement:
#
# "This product includes software developed by Digital Creations
# for use in the Z Object Publishing Environment
# (http://www.zope.org/)."
#
# In the event that the product being advertised includes an
# intact Zope distribution (with copyright and license included)
# then this clause is waived.
#
# 5. Names associated with Zope or Digital Creations must not be used to
# endorse or promote products derived from this software without
# prior written permission from Digital Creations.
#
# 6. Modified redistributions of any form whatsoever must retain
# the following acknowledgment:
#
# "This product includes software developed by Digital Creations
# for use in the Z Object Publishing Environment
# (http://www.zope.org/)."
#
# Intact (re-)distributions of any official Zope release do not
# require an external acknowledgement.
#
# 7. Modifications are encouraged but must be packaged separately as
# patches to official Zope releases. Distributions that do not
# clearly separate the patches from the original work must be clearly
# labeled as unofficial distributions. Modifications which do not
# carry the name Zope may be packaged in any form, as long as they
# conform to all of the clauses above.
# #
# 5. Support for software is included for 90 days in email only. Further
# support can be purchased separately.
# #
# Disclaimer # Disclaimer
# #
...@@ -39,12 +75,18 @@ ...@@ -39,12 +75,18 @@
# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT # OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
# SUCH DAMAGE. # SUCH DAMAGE.
##################################################################### #
#
# This software consists of contributions made by Digital Creations and
# many individuals on behalf of Digital Creations. Specific
# attributions are listed in the accompanying credits file.
#
##############################################################################
"""Start the server storage. """Start the server storage.
""" """
__version__ = "$Revision: 1.4 $"[11:-2] __version__ = "$Revision: 1.5 $"[11:-2]
import sys, os, getopt, string import sys, os, getopt, string
...@@ -77,30 +119,46 @@ def main(argv): ...@@ -77,30 +119,46 @@ def main(argv):
os.path.join(INSTANCE_HOME, 'var', 'ZEO_SERVER.pid') os.path.join(INSTANCE_HOME, 'var', 'ZEO_SERVER.pid')
) )
opts, args = getopt.getopt(args, 'p:Dh:') opts, args = getopt.getopt(args, 'p:Dh:U:Z:')
fs=os.path.join(INSTANCE_HOME, 'var', 'Data.fs') fs=os.path.join(INSTANCE_HOME, 'var', 'Data.fs')
usage="""%s -p port [options] [filename] usage="""%s [options] [filename]
where options are: where options are:
-D -- Run in debug mode -D -- Run in debug mode
-U -- Unix-domain socket file to listen on
-p -- port to listen on
-h -- host address to listen on -h -- host address to listen on
-s -- Don't use zdeamon
if no file name is specified, then %s is used. if no file name is specified, then %s is used.
""" % (me, fs) """ % (me, fs)
port=None port=None
debug=0 debug=0
host='' host=''
unix=None
Z=1
for o, v in opts: for o, v in opts:
if o=='-p': port=string.atoi(v) if o=='-p': port=string.atoi(v)
elif o=='-h': host=v elif o=='-h': host=v
elif o=='-U': unix=v
elif o=='-D': debug=1 elif o=='-D': debug=1
elif o=='-s': Z=0
if port is None:
try:
from ZServer.medusa import asyncore
sys.modules['asyncore']=asyncore
except: pass
if port is None and unix is None:
print usage print usage
print 'No port specified.' print 'No port specified.'
sys.exit(1) sys.exit(1)
...@@ -115,18 +173,20 @@ def main(argv): ...@@ -115,18 +173,20 @@ def main(argv):
__builtins__.__debug__=debug __builtins__.__debug__=debug
if debug: os.environ['Z_DEBUG_MODE']='1' if debug: os.environ['Z_DEBUG_MODE']='1'
try: import posix if Z:
except: pass try: import posix
else: except: pass
import zdaemon else:
zdaemon.run(sys.argv, '') import zdaemon
zdaemon.run(sys.argv, '')
import ZEO.StorageServer, ZODB.FileStorage, asyncore, zLOG import ZEO.StorageServer, ZODB.FileStorage, asyncore, zLOG
zLOG.LOG('ZEO Server', zLOG.INFO, 'Serving %s' % fs) zLOG.LOG('ZEO Server', zLOG.INFO, 'Serving %s' % fs)
if not unix: unix=host, port
ZEO.StorageServer.StorageServer( ZEO.StorageServer.StorageServer(
(host,port), unix,
{ {
'1': ZODB.FileStorage.FileStorage(fs) '1': ZODB.FileStorage.FileStorage(fs)
}, },
......
src/ZEO/zrpc.py
View file @ 5c2fe859
This diff is collapsed.
src/ZEO/zrpc/smac.py
View file @ 5c2fe859
###################################################################### ##############################################################################
# Digital Creations Options License Version 0.9.0
# -----------------------------------------------
# #
# Copyright (c) 1999, Digital Creations. All rights reserved. # Zope Public License (ZPL) Version 1.0
# -------------------------------------
# #
# This license covers Zope software delivered as "options" by Digital # Copyright (c) Digital Creations. All rights reserved.
# Creations.
# #
# Use in source and binary forms, with or without modification, are # This license has been certified as Open Source(tm).
# permitted provided that the following conditions are met:
# #
# 1. Redistributions are not permitted in any form. # Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
# #
# 2. This license permits one copy of software to be used by up to five # 1. Redistributions in source code must retain the above copyright
# developers in a single company. Use by more than five developers # notice, this list of conditions, and the following disclaimer.
# requires additional licenses.
# #
# 3. Software may be used to operate any type of website, including # 2. Redistributions in binary form must reproduce the above copyright
# publicly accessible ones. # notice, this list of conditions, and the following disclaimer in
# the documentation and/or other materials provided with the
# distribution.
# #
# 4. Software is not fully documented, and the customer acknowledges # 3. Digital Creations requests that attribution be given to Zope
# that the product can best be utilized by reading the source code. # in any manner possible. Zope includes a "Powered by Zope"
# button that is installed by default. While it is not a license
# violation to remove this button, it is requested that the
# attribution remain. A significant investment has been put
# into Zope, and this effort will continue if the Zope community
# continues to grow. This is one way to assure that growth.
#
# 4. All advertising materials and documentation mentioning
# features derived from or use of this software must display
# the following acknowledgement:
#
# "This product includes software developed by Digital Creations
# for use in the Z Object Publishing Environment
# (http://www.zope.org/)."
#
# In the event that the product being advertised includes an
# intact Zope distribution (with copyright and license included)
# then this clause is waived.
#
# 5. Names associated with Zope or Digital Creations must not be used to
# endorse or promote products derived from this software without
# prior written permission from Digital Creations.
#
# 6. Modified redistributions of any form whatsoever must retain
# the following acknowledgment:
#
# "This product includes software developed by Digital Creations
# for use in the Z Object Publishing Environment
# (http://www.zope.org/)."
#
# Intact (re-)distributions of any official Zope release do not
# require an external acknowledgement.
#
# 7. Modifications are encouraged but must be packaged separately as
# patches to official Zope releases. Distributions that do not
# clearly separate the patches from the original work must be clearly
# labeled as unofficial distributions. Modifications which do not
# carry the name Zope may be packaged in any form, as long as they
# conform to all of the clauses above.
# #
# 5. Support for software is included for 90 days in email only. Further
# support can be purchased separately.
# #
# Disclaimer # Disclaimer
# #
...@@ -39,79 +75,118 @@ ...@@ -39,79 +75,118 @@
# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT # OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
# SUCH DAMAGE. # SUCH DAMAGE.
###################################################################### #
#
# This software consists of contributions made by Digital Creations and
# many individuals on behalf of Digital Creations. Specific
# attributions are listed in the accompanying credits file.
#
##############################################################################
"""Sized message async connections """Sized message async connections
""" """
__version__ = "$Revision: 1.6 $"[11:-2] __version__ = "$Revision: 1.7 $"[11:-2]
import asyncore, string, struct, zLOG, sys, Acquisition
from zLOG import LOG, TRACE, ERROR
import asyncore, string, struct, zLOG class SizedMessageAsyncConnection(Acquisition.Explicit, asyncore.dispatcher):
from zLOG import LOG, INFO, ERROR
class SizedMessageAsyncConnection(asyncore.dispatcher): __append=None # Marker indicating that we're closed
def __init__(self, sock, addr): socket=None # to outwit Sam's getattr
asyncore.dispatcher.__init__(self, sock)
def __init__(self, sock, addr, map=None, debug=None):
SizedMessageAsyncConnection.inheritedAttribute(
'__init__')(self, sock, map)
self.addr=addr self.addr=addr
if debug is not None:
self._debug=debug
elif not hasattr(self, '_debug'):
self._debug=__debug__ and 'smac'
self.__state=None self.__state=None
self.__inp=None self.__inp=None
self.__inpl=0
self.__l=4 self.__l=4
self.__output=output=[] self.__output=output=[]
self.__append=output.append self.__append=output.append
self.__pop=output.pop self.__pop=output.pop
def handle_read(self, def handle_read(self,
join=string.join, StringType=type('')): join=string.join, StringType=type(''), _type=type,
l=self.__l StringType=type(''), _None=None):
d=self.recv(l)
d=self.recv(8096)
if not d: return
inp=self.__inp inp=self.__inp
if inp is None: if inp is _None:
inp=d inp=d
elif type(inp) is StringType: elif _type(inp) is StringType:
inp=[inp,d] inp=[inp,d]
else: else:
inp.append(d) inp.append(d)
l=l-len(d) inpl=self.__inpl+len(d)
if l <= 0: l=self.__l
if type(inp) is not StringType: inp=join(inp,'')
if self.__state is None: while 1:
# waiting for message
self.__l=struct.unpack(">i",inp)[0] if l <= inpl:
self.__state=1 # Woo hoo, we have enough data
self.__inp=None if _type(inp) is not StringType: inp=join(inp,'')
d=inp[:l]
inp=inp[l:]
inpl=inpl-l
if self.__state is _None:
# waiting for message
l=struct.unpack(">i",d)[0]
self.__state=1
else:
l=4
self.__state=_None
self.message_input(d)
else: else:
self.__inp=None break # not enough data
self.__l=4
self.__state=None self.__l=l
self.message_input(inp) self.__inp=inp
else: self.__inpl=inpl
self.__l=l
self.__inp=inp
def readable(self): return 1 def readable(self): return 1
def writable(self): return not not self.__output def writable(self): return not not self.__output
def handle_write(self): def handle_write(self):
output=self.__output output=self.__output
if output: while output:
v=output[0] v=output[0]
n=self.send(v) n=self.send(v)
if n < len(v): if n < len(v):
output[0]=v[n:] output[0]=v[n:]
break # we can't write any more
else: else:
del output[0] del output[0]
#break # waaa
def handle_close(self): def handle_close(self):
self.close() self.close()
def message_output(self, message, def message_output(self, message,
pack=struct.pack, len=len): pack=struct.pack, len=len):
if __debug__: if self._debug:
if len(message) > 40: m=message[:40]+' ...' if len(message) > 40: m=message[:40]+' ...'
else: m=message else: m=message
LOG('smax', INFO, 'message_output %s' % `m`) LOG(self._debug, TRACE, 'message_output %s' % `m`)
self.__append(pack(">i",len(message))+message)
append=self.__append
if append is None:
raise Disconnected, (
"This action is temporarily unavailable."
"<p>"
)
append(pack(">i",len(message))+message)
def log_info(self, message, type='info'): def log_info(self, message, type='info'):
if type=='error': type=ERROR if type=='error': type=ERROR
...@@ -119,3 +194,13 @@ class SizedMessageAsyncConnection(asyncore.dispatcher): ...@@ -119,3 +194,13 @@ class SizedMessageAsyncConnection(asyncore.dispatcher):
LOG('ZEO', type, message) LOG('ZEO', type, message)
log=log_info log=log_info
def close(self):
if self.__append is not None:
self.__append=None
SizedMessageAsyncConnection.inheritedAttribute('close')(self)
class Disconnected(Exception):
"""The client has become disconnected from the server
"""
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7