Use tagged version of ZConfig

trunk/src/ZConfig/BRANCHES.txt

+Branches defined specifically for the ZConfig package are listed
+below.  The canonical version of this list is on the HEAD of the
+ZConfig package.
+
+zconfig-brace-syntax
+    NOT ACTIVE
+    An example of an alternate syntax for ZConfig.  This syntax was
+    developed while trying the package initially, but was rejected.
+    It is saved on a branch to avoid losing historical information.
+
+zconfig-schema-devel-branch
+    NOT ACTIVE
+    Development branch for schema support in ZConfig.  The branch is
+    based on the ZConfig trunk, but the development is strongly based
+    on the work Chris McDonough started in the chrism-install-branch
+    for Zope 2.  This was merged into the trunk on 3-Jan-2002.

trunk/src/ZConfig/NEWS.txt

+Changes since ZConfig 2.1:
+
+- More documentation has been written.
+
+- Added a timedelta datatype function; the input is the same as for
+  the time-interval datatype, but the resulting value is a
+  datetime.timedelta object.
+
+- Make sure keys specified as attributes of the <default> element are
+  converted by the appropriate key type, and are re-checked for
+  derived sections.
+
+- Refactored the ZConfig.components.logger schema components so that a
+  schema can import just one of the "eventlog" or "logger" sections if
+  desired.  This can be helpful to avoid naming conflicts.
+
+- Added a reopen() method to the logger factories.
+
+- Always use an absolute pathname when opening a FileHandler.
+
+- A fix to the logger 'format' key to allow the %(process)d expansion variable
+  that the logging package supports.
+
+- A new timedelta built-in datatype was added.  Similar to time-interval
+  except that it returns a datetime.timedelta object instead.
+
+Changes since ZConfig 2.0:
+
+- Removed compatibility with Python 2.1 and 2.2.
+
+- Schema components must really be in Python packages; the directory
+  search has been modified to perform an import to locate the package
+  rather than incorrectly implementing the search algorithm.
+
+- The default objects use for section values now provide a method
+  getSectionAttributes(); this returns a list of all the attributes of
+  the section object which store configuration-defined data (including
+  information derived from the schema).
+
+- Default information can now be included in a schema for <key
+  name="+"> and <multikey name="+"> by using <default key="...">.
+
+- More documentation has been added to discuss schema extension.
+
+- Support for a Unicode-free Python has been fixed.
+
+- Derived section types now inherit the datatype of the base type if
+  no datatype is identified explicitly.
+
+- Derived section types can now override the keytype instead of always
+  inheriting from their base type.
+
+- <import package='...'/> makes use of the current prefix if the
+  package name begins witha dot.
+
+- Added two standard datatypes:  dotted-name and dotted-suffix.
+
+- Added two standard schema components: ZConfig.components.basic and
+  ZConfig.components.logger.
+
+Changes since ZConfig 1.0:
+
+- Configurations can import additional schema components using a new
+  "%import" directive; this can be used to integrate 3rd-party
+  components into an application.
+
+- Schemas may be extended using a new "extends" attribute on the
+  <schema> element.
+
+- Better error messages when elements in a schema definition are
+  improperly nested.
+
+- The "zconfig" script can now simply verify that a schema definition
+  is valid, if that's all that's needed.

trunk/src/ZConfig/PACKAGE.cfg

+# Load the license from an external source, so we don't have to keep a
+# copy of it sitting around:
+<load>
+  LICENSE.txt  http://cvs.zope.org/Zope3/ZopePublicLicense.txt?rev=HEAD
+</load>
+
+# Add a few things to the distribution root.
+<distribution>
+  doc
+  LICENSE.txt
+  NEWS.txt
+  README.txt
+</distribution>
+
+# Specify what is included in the component.
+<collection>
+  # Python modules from the package:
+  *.py
+
+  # Child packages:
+  components
+  tests
+
+  # Other files and directories needed when distutils runs:
+  scripts
+</collection>

trunk/src/ZConfig/PUBLICATION.cfg

+Metadata-Version: 1.0
+Name: ZConfig
+Summary: Structured Configuration Library
+Home-page: http://www.zope.org/Members/fdrake/zconfig/
+Author: Fred L. Drake, Jr.
+Author-email: fred@zope.com
+License: ZPL 2
+Description: ZConfig is a configuration library intended for general use.  It
+        supports a hierarchical schema-driven configuration model that allows
+        a schema to specify data conversion routines written in Python.
+        ZConfig's model is very different from the model supported by the
+        ConfigParser module found in Python's standard library, and is more
+        suitable to configuration-intensive applications.
+        
+        ZConfig schema are written in an XML-based language and are able to
+        "import" schema components provided by Python packages.  Since
+        components are able to bind to conversion functions provided by Python
+        code in the package (or elsewhere), configuration objects can be
+        arbitrarily complex, with values that have been verified against
+        arbitrary constraints.  This makes it easy for applications to
+        separate configuration support from configuration loading even with
+        configuration data being defined and consumed by a wide range of
+        separate packages.
+Platform: POSIX
+Platform: Windows
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: System Administrators
+Classifier: License :: OSI Approved :: Zope Public License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python
+Classifier: Topic :: Software Development :: Libraries :: Python Modules

trunk/src/ZConfig/README.txt

+This is ZConfig.
+
+ZConfig is a configuration library intended for general use.  It
+supports a hierarchical schema-driven configuration model that allows
+a schema to specify data conversion routines written in Python.
+ZConfig's model is very different from the model supported by the
+ConfigParser module found in Python's standard library, and is more
+suitable to configuration-intensive applications.
+
+ZConfig schema are written in an XML-based language and are able to
+"import" schema components provided by Python packages.  Since
+components are able to bind to conversion functions provided by Python
+code in the package (or elsewhere), configuration objects can be
+arbitrarily complex, with values that have been verified against
+arbitrary constraints.  This makes it easy for applications to
+separate configuration support from configuration loading even with
+configuration data being defined and consumed by a wide range of
+separate packages.
+
+ZConfig is licensed under the Zope Public License, version 2.0.  See
+the file LICENSE.txt in the distribution for the full license text.
+
+Reference documentation is available in the ZConfig/doc/ directory.
+
+Information on the latest released version of the ZConfig package is
+available at
+
+  http://www.zope.org/Members/fdrake/zconfig/
+
+You may either create an RPM and install this, or install directly from
+the source distribution.
+
+
+Creating RPMS:
+
+  python setup.py bdist_rpm
+
+  If you need to force the Python interpreter to, for example, python2:
+
+    python2 setup.py bdist_rpm --python=python2
+
+
+Installation from the source distribution:
+
+    python setup.py install
+
+  To install to a user's home-dir:
+    python setup.py install --home=<dir>
+
+  To install to another prefix (eg. /usr/local)
+    python setup.py install --prefix=/usr/local
+
+  If you need to force the python interpreter to eg. python2:
+    python2 setup.py install
+
+  For more information please refer to
+    http://www.python.org/doc/current/inst/inst.html

trunk/src/ZConfig/SETUP.cfg

+# Metadata used by zpkg (mostly a sample for testing).
+
+#documentation  doc/zconfig.pdf
+#documentation  doc/schema.dtd
+
+script scripts/zconfig*

trunk/src/ZConfig/__init__.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Configuration data structures and loader for the ZRS.
+
+$Id: __init__.py,v 1.18 2004/04/15 20:33:32 fdrake Exp $
+"""
+version_info = (2, 2)
+__version__ = ".".join([str(n) for n in version_info])
+
+from ZConfig.loader import loadConfig, loadConfigFile
+from ZConfig.loader import loadSchema, loadSchemaFile
+
+
+class ConfigurationError(Exception):
+    """Base class for ZConfig exceptions."""
+
+    def __init__(self, msg, url=None):
+        self.message = msg
+        self.url = url
+        Exception.__init__(self, msg)
+
+    def __str__(self):
+        return self.message
+
+
+class _ParseError(ConfigurationError):
+    def __init__(self, msg, url, lineno, colno=None):
+        self.lineno = lineno
+        self.colno = colno
+        ConfigurationError.__init__(self, msg, url)
+
+    def __str__(self):
+        s = self.message
+        if self.url:
+            s += "\n("
+        elif (self.lineno, self.colno) != (None, None):
+            s += " ("
+        if self.lineno:
+            s += "line %d" % self.lineno
+            if self.colno is not None:
+                s += ", column %d" % self.colno
+            if self.url:
+                s += " in %s)" % self.url
+            else:
+                s += ")"
+        elif self.url:
+            s += self.url + ")"
+        return s
+
+
+class SchemaError(_ParseError):
+    """Raised when there's an error in the schema itself."""
+
+    def __init__(self, msg, url=None, lineno=None, colno=None):
+        _ParseError.__init__(self, msg, url, lineno, colno)
+
+
+class SchemaResourceError(SchemaError):
+    """Raised when there's an error locating a resource required by the schema.
+    """
+
+    def __init__(self, msg, url=None, lineno=None, colno=None,
+                 path=None, package=None, filename=None):
+        self.filename = filename
+        self.package = package
+        if path is not None:
+            path = path[:]
+        self.path = path
+        SchemaError.__init__(self, msg, url, lineno, colno)
+
+    def __str__(self):
+        s = SchemaError.__str__(self)
+        if self.package is not None:
+            s += "\n  Package name: " + repr(self.package)
+        if self.filename is not None:
+            s += "\n  File name: " + repr(self.filename)
+        if self.package is not None:
+            s += "\n  Package path: " + repr(self.path)
+        return s
+
+
+class ConfigurationSyntaxError(_ParseError):
+    """Raised when there's a syntax error in a configuration file."""
+
+
+class DataConversionError(ConfigurationError, ValueError):
+    """Raised when a data type conversion function raises ValueError."""
+
+    def __init__(self, exception, value, position):
+        ConfigurationError.__init__(self, str(exception))
+        self.exception = exception
+        self.value = value
+        self.lineno, self.colno, self.url = position
+
+    def __str__(self):
+        s = "%s (line %s" % (self.message, self.lineno)
+        if self.colno is not None:
+            s += ", %s" % self.colno
+        if self.url:
+            s += ", in %s)" % self.url
+        else:
+            s += ")"
+        return s
+
+
+class SubstitutionSyntaxError(ConfigurationError):
+    """Raised when interpolation source text contains syntactical errors."""
+
+
+class SubstitutionReplacementError(ConfigurationSyntaxError, LookupError):
+    """Raised when no replacement is available for a reference."""
+
+    def __init__(self, source, name, url=None, lineno=None):
+        self.source = source
+        self.name = name
+        ConfigurationSyntaxError.__init__(
+            self, "no replacement for " + `name`, url, lineno)

trunk/src/ZConfig/cfgparser.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Configuration parser."""
+
+import ZConfig
+import ZConfig.url
+
+from ZConfig.substitution import isname, substitute
+
+
+class ZConfigParser:
+    __metaclass__ = type
+    __slots__ = ('resource', 'context', 'lineno',
+                 'stack', 'defs', 'file', 'url')
+
+    def __init__(self, resource, context, defines=None):
+        self.resource = resource
+        self.context = context
+        self.file = resource.file
+        self.url = resource.url
+        self.lineno = 0
+        self.stack = []   # [(type, name, prevmatcher), ...]
+        if defines is None:
+            defines = {}
+        self.defs = defines
+
+    def nextline(self):
+        line = self.file.readline()
+        if line:
+            self.lineno += 1
+            return False, line.strip()
+        else:
+            return True, None
+
+    def parse(self, section):
+        done, line = self.nextline()
+        while not done:
+            if line[:1] in ("", "#"):
+                # blank line or comment
+                pass
+
+            elif line[:2] == "</":
+                # section end
+                if line[-1] != ">":
+                    self.error("malformed section end")
+                section = self.end_section(section, line[2:-1])
+
+            elif line[0] == "<":
+                # section start
+                if line[-1] != ">":
+                    self.error("malformed section start")
+                section = self.start_section(section, line[1:-1])
+
+            elif line[0] == "%":
+                self.handle_directive(section, line[1:])
+
+            else:
+                self.handle_key_value(section, line)
+
+            done, line = self.nextline()
+
+        if self.stack:
+            self.error("unclosed sections not allowed")
+
+    def start_section(self, section, rest):
+        isempty = rest[-1:] == "/"
+        if isempty:
+            text = rest[:-1].rstrip()
+        else:
+            text = rest.rstrip()
+        # parse section start stuff here
+        m = _section_start_rx.match(text)
+        if not m:
+            self.error("malformed section header")
+        type, name = m.group('type', 'name')
+        type = type.lower()
+        if name:
+            name = name.lower()
+        try:
+            newsect = self.context.startSection(section, type, name)
+        except ZConfig.ConfigurationError, e:
+            self.error(e[0])
+
+        if isempty:
+            self.context.endSection(section, type, name, newsect)
+            return section
+        else:
+            self.stack.append((type, name, section))
+            return newsect
+
+    def end_section(self, section, rest):
+        if not self.stack:
+            self.error("unexpected section end")
+        type = rest.rstrip().lower()
+        opentype, name, prevsection = self.stack.pop()
+        if type != opentype:
+            self.error("unbalanced section end")
+        try:
+            self.context.endSection(
+                prevsection, type, name, section)
+        except ZConfig.ConfigurationError, e:
+            self.error(e[0])
+        return prevsection
+
+    def handle_key_value(self, section, rest):
+        m = _keyvalue_rx.match(rest)
+        if not m:
+            self.error("malformed configuration data")
+        key, value = m.group('key', 'value')
+        if not value:
+            value = ''
+        else:
+            value = self.replace(value)
+        try:
+            section.addValue(key, value, (self.lineno, None, self.url))
+        except ZConfig.ConfigurationError, e:
+            self.error(e[0])
+
+    def handle_directive(self, section, rest):
+        m = _keyvalue_rx.match(rest)
+        if not m:
+            self.error("missing or unrecognized directive")
+        name, arg = m.group('key', 'value')
+        if name not in ("define", "import", "include"):
+            self.error("unknown directive: " + `name`)
+        if not arg:
+            self.error("missing argument to %%%s directive" % name)
+        if name == "include":
+            self.handle_include(section, arg)
+        elif name == "define":
+            self.handle_define(section, arg)
+        elif name == "import":
+            self.handle_import(section, arg)
+        else:
+            assert 0, "unexpected directive for " + `"%" + rest`
+
+    def handle_import(self, section, rest):
+        pkgname = self.replace(rest.strip())
+        self.context.importSchemaComponent(pkgname)
+
+    def handle_include(self, section, rest):
+        rest = self.replace(rest.strip())
+        newurl = ZConfig.url.urljoin(self.url, rest)
+        self.context.includeConfiguration(section, newurl, self.defs)
+
+    def handle_define(self, section, rest):
+        parts = rest.split(None, 1)
+        defname = parts[0].lower()
+        defvalue = ''
+        if len(parts) == 2:
+            defvalue = parts[1]
+        if self.defs.has_key(defname):
+            self.error("cannot redefine " + `defname`)
+        if not isname(defname):
+            self.error("not a substitution legal name: " + `defname`)
+        self.defs[defname] = self.replace(defvalue)
+
+    def replace(self, text):
+        try:
+            return substitute(text, self.defs)
+        except ZConfig.SubstitutionReplacementError, e:
+            e.lineno = self.lineno
+            e.url = self.url
+            raise
+
+    def error(self, message):
+        raise ZConfig.ConfigurationSyntaxError(message, self.url, self.lineno)
+
+
+import re
+# _name_re does not allow "(" or ")" for historical reasons.  Though
+# the restriction could be lifted, there seems no need to do so.
+_name_re = r"[^\s()]+"
+_keyvalue_rx = re.compile(r"(?P<key>%s)\s*(?P<value>[^\s].*)?$"
+                          % _name_re)
+_section_start_rx = re.compile(r"(?P<type>%s)"
+                               r"(?:\s+(?P<name>%s))?"
+                               r"$"
+                               % (_name_re, _name_re))
+del re

trunk/src/ZConfig/cmdline.py

+##############################################################################
+#
+# Copyright (c) 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+"""Support for command-line provision of settings.
+
+This module provides an extension of the ConfigLoader class which adds
+a way to add configuration settings from an alternate source.  Each
+setting is described by a string of the form::
+
+    some/path/to/key=value
+"""
+
+import ZConfig
+import ZConfig.loader
+import ZConfig.matcher
+
+
+class ExtendedConfigLoader(ZConfig.loader.ConfigLoader):
+    def __init__(self, schema):
+        ZConfig.loader.ConfigLoader.__init__(self, schema)
+        self.clopts = []   # [(optpath, value, source-position), ...]
+
+    def addOption(self, spec, pos=None):
+        if pos is None:
+            pos = "<command-line option>", -1, -1
+        if "=" not in spec:
+            e = ZConfig.ConfigurationSyntaxError(
+                "invalid configuration specifier", *pos)
+            e.specifier = spec
+            raise e
+        # For now, just add it to the list; not clear that checking
+        # against the schema at this point buys anything.
+        opt, val = spec.split("=", 1)
+        optpath = opt.split("/")
+        if "" in optpath:
+            # // is not allowed in option path
+            e = ZConfig.ConfigurationSyntaxError(
+                "'//' is not allowed in an option path", *pos)
+            e.specifier = spec
+            raise e
+        self.clopts.append((optpath, val, pos))
+
+    def createSchemaMatcher(self):
+        if self.clopts:
+            sm = ExtendedSchemaMatcher(self.schema)
+            sm.set_optionbag(self.cook())
+        else:
+            sm = ZConfig.loader.ConfigLoader.createSchemaMatcher(self)
+        return sm
+
+    def cook(self):
+        if self.clopts:
+            return OptionBag(self.schema, self.schema, self.clopts)
+        else:
+            return None
+
+
+class OptionBag:
+    def __init__(self, schema, sectiontype, options):
+        self.sectiontype = sectiontype
+        self.schema = schema
+        self.keypairs = {}
+        self.sectitems = []
+        self._basic_key = schema.registry.get("basic-key")
+        for item in options:
+            optpath, val, pos = item
+            name = sectiontype.keytype(optpath[0])
+            if len(optpath) == 1:
+                self.add_value(name, val, pos)
+            else:
+                self.sectitems.append(item)
+
+    def basic_key(self, s, pos):
+        try:
+            return self._basic_key(s)
+        except ValueError:
+            raise ZConfig.ConfigurationSyntaxError(
+                "could not convert basic-key value", *pos)
+
+    def add_value(self, name, val, pos):
+        if self.keypairs.has_key(name):
+            L = self.keypairs[name]
+        else:
+            L = []
+            self.keypairs[name] = L
+        L.append((val, pos))
+
+    def has_key(self, name):
+        return self.keypairs.has_key(name)
+
+    def get_key(self, name):
+        """Return a list of (value, pos) items for the key 'name'.
+
+        The returned list may be empty.
+        """
+        L = self.keypairs.get(name)
+        if L:
+            del self.keypairs[name]
+            return L
+        else:
+            return []
+
+    def keys(self):
+        return self.keypairs.keys()
+
+    def get_section_info(self, type, name):
+        L = []  # what pertains to the child section
+        R = []  # what we keep
+        for item in self.sectitems:
+            optpath, val, pos = item
+            s = optpath[0]
+            bk = self.basic_key(s, pos)
+            if name and s.lower() == name:
+                L.append((optpath[1:], val, pos))
+            elif bk == type:
+                L.append((optpath[1:], val, pos))
+            else:
+                R.append(item)
+        if L:
+            self.sectitems[:] = R
+            return OptionBag(self.schema, self.schema.gettype(type), L)
+        else:
+            return None
+
+    def finish(self):
+        if self.sectitems or self.keypairs:
+            raise ZConfig.ConfigurationError(
+                "not all command line options were consumed")
+
+
+class MatcherMixin:
+    def set_optionbag(self, bag):
+        self.optionbag = bag
+
+    def addValue(self, key, value, position):
+        try:
+            realkey = self.type.keytype(key)
+        except ValueError, e:
+            raise ZConfig.DataConversionError(e, key, position)
+        if self.optionbag.has_key(realkey):
+            return
+        ZConfig.matcher.BaseMatcher.addValue(self, key, value, position)
+
+    def createChildMatcher(self, type, name):
+        sm = ZConfig.matcher.BaseMatcher.createChildMatcher(self, type, name)
+        bag = self.optionbag.get_section_info(type.name, name)
+        if bag is not None:
+            sm = ExtendedSectionMatcher(
+                sm.info, sm.type, sm.name, sm.handlers)
+            sm.set_optionbag(bag)
+        return sm
+
+    def finish_optionbag(self):
+        for key in self.optionbag.keys():
+            for val, pos in self.optionbag.get_key(key):
+                ZConfig.matcher.BaseMatcher.addValue(self, key, val, pos)
+        self.optionbag.finish()
+
+
+class ExtendedSectionMatcher(MatcherMixin, ZConfig.matcher.SectionMatcher):
+    def finish(self):
+        self.finish_optionbag()
+        return ZConfig.matcher.SectionMatcher.finish(self)
+
+class ExtendedSchemaMatcher(MatcherMixin, ZConfig.matcher.SchemaMatcher):
+    def finish(self):
+        self.finish_optionbag()
+        return ZConfig.matcher.SchemaMatcher.finish(self)

trunk/src/ZConfig/components/__init__.py

+# This is a Python package.

trunk/src/ZConfig/components/basic/__init__.py

+# This is a Python package.

trunk/src/ZConfig/components/basic/component.xml

+<component>
+  <description>
+    Convenient loader which causes all the "basic" components to be
+    loaded.
+  </description>
+
+  <import package="ZConfig.components.basic" file="mapping.xml"/>
+
+</component>

trunk/src/ZConfig/components/basic/mapping.py

+##############################################################################
+#
+# Copyright (c) 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+"""Python datatype for the ZConfig.components.basic.mapping section type."""
+
+def mapping(section):
+    return section.mapping

trunk/src/ZConfig/components/basic/mapping.xml

+<component>
+
+  <sectiontype name="ZConfig.basic.mapping"
+               datatype="ZConfig.components.basic.mapping.mapping">
+    <description>
+      Section that provides a simple mapping implementation.  An
+      application should derive a more specific section type for use
+      in configuration files:
+
+        &lt;import package="ZConfig.components.basic"
+                     file="mapping.xml"
+                     /&gt;
+
+        &lt;sectiontype name="mapping"
+                     extends="ZConfig.basic.mapping"
+                     /&gt;
+
+      If a non-standard keytype is needed, it can be overridden as
+      well:
+
+        &lt;sectiontype name="system-map"
+                     extends="ZConfig.basic.mapping"
+                     keytype="mypkg.datatypes.system_name"
+                     /&gt;
+
+    </description>
+
+    <key name="+"
+         attribute="mapping"
+         required="no"
+         />
+  </sectiontype>
+
+</component>

trunk/src/ZConfig/components/basic/tests/__init__.py

+# This is a Python package.

trunk/src/ZConfig/components/basic/tests/test_mapping.py

+##############################################################################
+#
+# Copyright (c) 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+"""Tests of the 'basic' section types provided as part of
+ZConfig.components.basic."""
+
+import unittest
+
+from ZConfig.tests import support
+
+
+SIMPLE_SCHEMA = '''\
+<schema>
+  <import package="ZConfig.components.basic" file="mapping.xml" />
+
+  <sectiontype name="dict"
+               extends="ZConfig.basic.mapping" />
+
+  <sectiontype name="intkeys"
+               extends="ZConfig.basic.mapping"
+               keytype="integer" />
+
+  <section name="*"
+           type="dict"
+           attribute="simple_dict" />
+
+  <section name="*"
+           type="intkeys"
+           attribute="int_dict" />
+
+</schema>
+'''
+
+
+class BasicSectionTypeTestCase(support.TestBase):
+    schema = None
+
+    def setUp(self):
+        if self.schema is None:
+            self.__class__.schema = self.load_schema_text(SIMPLE_SCHEMA)
+
+    def test_simple_empty_dict(self):
+        conf = self.load_config_text(self.schema, "<dict/>")
+        self.assertEqual(conf.simple_dict, {})
+        conf = self.load_config_text(self.schema, """\
+            <dict foo>
+            # comment
+            </dict>
+            """)
+        self.assertEqual(conf.simple_dict, {})
+
+    def test_simple_dict(self):
+        conf = self.load_config_text(self.schema, """\
+           <dict foo>
+           key-one value-one
+           key-two value-two
+           </dict>
+           """)
+        L = conf.simple_dict.items()
+        L.sort()
+        self.assertEqual(L, [("key-one", "value-one"),
+                             ("key-two", "value-two")])
+
+    def test_derived_dict(self):
+        conf = self.load_config_text(self.schema, """\
+            <intkeys>
+            1 foo
+            2 bar
+            42 question?
+            </intkeys>
+            """)
+        L = conf.int_dict.items()
+        L.sort()
+        self.assertEqual(L, [(1, "foo"), (2, "bar"), (42, "question?")])
+
+
+def test_suite():
+    return unittest.makeSuite(BasicSectionTypeTestCase)

trunk/src/ZConfig/components/logger/__init__.py

+##############################################################################
+#
+# Copyright (c) 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""ZConfig schema component package for logging configuration."""
+
+# Make sure we can't import this if "logging" isn't available; we
+# don't want partial imports to appear to succeed.
+
+try:
+    import logging
+except ImportError:
+    import sys
+    del sys.modules[__name__]

trunk/src/ZConfig/components/logger/abstract.xml

+<component>
+  <description>
+  </description>
+
+  <abstracttype name="ZConfig.logger.handler"/>
+  <abstracttype name="ZConfig.logger.log"/>
+</component>

trunk/src/ZConfig/components/logger/base-logger.xml

+<component prefix="ZConfig.components.logger.logger">
+
+  <import package="ZConfig.components.logger" file="abstract.xml"/>
+
+  <sectiontype name="ZConfig.logger.base-logger">
+    <description>
+      Base definition for the logger types defined by
+      ZConfig.components.logger.  This exists entirely to provide
+      shared key definitions and documentation.
+    </description>
+
+    <key name="level"
+         datatype="ZConfig.components.logger.datatypes.logging_level"
+         default="info">
+      <description>
+        Verbosity setting for the logger.  Values must be a name of
+        a level, or an integer in the range [0..50].  The names of the
+        levels, in order of increasing verbosity (names on the same
+        line are equivalent):
+
+            critical, fatal
+            error
+            warn, warning
+            info
+            blather
+            debug
+            trace
+            all
+
+        The special name "notset", or the numeric value 0, indicates
+        that the setting for the parent logger should be used.
+
+        It is strongly recommended that names be used rather than
+        numeric values to ensure that configuration files can be
+        deciphered more easily.
+      </description>
+    </key>
+
+    <multisection type="ZConfig.logger.handler"
+                  attribute="handlers" name="*">
+      <description>
+        Handlers to install on this logger.  Each handler describes
+        how logging events should be presented.
+      </description>
+    </multisection>
+  </sectiontype>
+
+</component>

trunk/src/ZConfig/components/logger/component.xml

+<component prefix="ZConfig.components.logger.datatypes">
+  <description>
+  </description>
+
+  <import package="ZConfig.components.logger" file="abstract.xml"/>
+  <import package="ZConfig.components.logger" file="handlers.xml"/>
+  <import package="ZConfig.components.logger" file="logger.xml"/>
+  <import package="ZConfig.components.logger" file="eventlog.xml"/>
+
+</component>

trunk/src/ZConfig/components/logger/datatypes.py

+##############################################################################
+#
+# Copyright (c) 2002 Zope Corporation and Contributors. All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+"""ZConfig datatypes for logging support."""
+
+
+_logging_levels = {
+    "critical": 50,
+    "fatal": 50,
+    "error": 40,
+    "warn": 30,
+    "warning": 30,
+    "info": 20,
+    "blather": 15,
+    "debug": 10,
+    "trace": 5,
+    "all": 1,
+    "notset": 0,
+    }
+
+def logging_level(value):
+    s = str(value).lower()
+    if _logging_levels.has_key(s):
+        return _logging_levels[s]
+    else:
+        v = int(s)
+        if v < 0 or v > 50:
+            raise ValueError("log level not in range: " + `v`)
+        return v

trunk/src/ZConfig/components/logger/eventlog.xml

+<component prefix="ZConfig.components.logger.logger">
+
+  <import package="ZConfig.components.logger" file="abstract.xml"/>
+  <import package="ZConfig.components.logger" file="base-logger.xml"/>
+
+  <sectiontype name="eventlog"
+               datatype=".EventLogFactory"
+               extends="ZConfig.logger.base-logger"
+               implements="ZConfig.logger.log">
+    <description>
+      Configuration for the root logger.
+    </description>
+  </sectiontype>
+
+</component>

trunk/src/ZConfig/components/logger/factory.py

+##############################################################################
+#
+# Copyright (c) 2002 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+_marker = object()
+
+class Factory:
+    """Generic wrapper for instance construction.
+
+    Calling the factory causes the instance to be created if it hasn't
+    already been created, and returns the object.  Calling the factory
+    multiple times returns the same object.
+
+    The instance is created using the factory's create() method, which
+    must be overriden by subclasses.
+    """
+    def __init__(self):
+        self.instance = _marker
+
+    def __call__(self):
+        if self.instance is _marker:
+            self.instance = self.create()
+        return self.instance
+
+    def create(self):
+        raise NotImplementedError("subclasses need to override create()")

trunk/src/ZConfig/components/logger/handlers.py

+##############################################################################
+#
+# Copyright (c) 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""ZConfig factory datatypes for log handlers."""
+
+import sys
+
+from ZConfig.components.logger.factory import Factory
+
+
+_log_format_variables = {
+    'name': '',
+    'levelno': '3',
+    'levelname': 'DEBUG',
+    'pathname': 'apath',
+    'filename': 'afile',
+    'module': 'amodule',
+    'lineno': 1,
+    'created': 1.1,
+    'asctime': 'atime',
+    'msecs': 1,
+    'relativeCreated': 1,
+    'thread': 1,
+    'message': 'amessage',
+    'process': 1,
+    }
+
+def log_format(value):
+    value = ctrl_char_insert(value)
+    try:
+        # Make sure the format string uses only names that will be
+        # provided, and has reasonable type flags for each, and does
+        # not expect positional args.
+        value % _log_format_variables
+    except (ValueError, KeyError):
+        raise ValueError, 'Invalid log format string %s' % value
+    return value
+
+_control_char_rewrites = {r'\n': '\n', r'\t': '\t', r'\b': '\b',
+                          r'\f': '\f', r'\r': '\r'}.items()
+
+def ctrl_char_insert(value):
+    for pattern, replacement in _control_char_rewrites:
+        value = value.replace(pattern, replacement)
+    return value
+
+
+class HandlerFactory(Factory):
+    def __init__(self, section):
+        Factory.__init__(self)
+        self.section = section
+
+    def create_loghandler(self):
+        raise NotImplementedError(
+            "subclasses must override create_loghandler()")
+
+    def create(self):
+        import logging
+        logger = self.create_loghandler()
+        logger.setFormatter(logging.Formatter(self.section.format,
+                                              self.section.dateformat))
+        logger.setLevel(self.section.level)
+        return logger
+
+    def getLevel(self):
+        return self.section.level
+
+class FileHandlerFactory(HandlerFactory):
+    def create_loghandler(self):
+        from ZConfig.components.logger import loghandler
+        path = self.section.path
+        if path == "STDERR":
+            handler = loghandler.StreamHandler(sys.stderr)
+        elif path == "STDOUT":
+            handler = loghandler.StreamHandler(sys.stdout)
+        else:
+            handler = loghandler.FileHandler(path)
+        return handler
+
+_syslog_facilities = {
+    "auth": 1,
+    "authpriv": 1,
+    "cron": 1,
+    "daemon": 1,
+    "kern": 1,
+    "lpr": 1,
+    "mail": 1,
+    "news": 1,
+    "security": 1,
+    "syslog": 1,
+    "user": 1,
+    "uucp": 1,
+    "local0": 1,
+    "local1": 1,
+    "local2": 1,
+    "local3": 1,
+    "local4": 1,
+    "local5": 1,
+    "local6": 1,
+    "local7": 1,
+    }
+
+def syslog_facility(value):
+    value = value.lower()
+    if not _syslog_facilities.has_key(value):
+        L = _syslog_facilities.keys()
+        L.sort()
+        raise ValueError("Syslog facility must be one of " + ", ".join(L))
+    return value
+
+class SyslogHandlerFactory(HandlerFactory):
+    def create_loghandler(self):
+        from ZConfig.components.logger import loghandler
+        return loghandler.SysLogHandler(self.section.address.address,
+                                        self.section.facility)
+
+class Win32EventLogFactory(HandlerFactory):
+    def create_loghandler(self):
+        from ZConfig.components.logger import loghandler
+        return loghandler.Win32EventLogHandler(self.section.appname)
+
+def http_handler_url(value):
+    import urlparse
+    scheme, netloc, path, param, query, fragment = urlparse.urlparse(value)
+    if scheme != 'http':
+        raise ValueError, 'url must be an http url'
+    if not netloc:
+        raise ValueError, 'url must specify a location'
+    if not path:
+        raise ValueError, 'url must specify a path'
+    q = []
+    if param:
+        q.append(';')
+        q.append(param)
+    if query:
+        q.append('?')
+        q.append(query)
+    if fragment:
+        q.append('#')
+        q.append(fragment)
+    return (netloc, path + ''.join(q))
+
+def get_or_post(value):
+    value = value.upper()
+    if value not in ('GET', 'POST'):
+        raise ValueError('method must be "GET" or "POST", instead received: '
+                         + repr(value))
+    return value
+
+class HTTPHandlerFactory(HandlerFactory):
+    def create_loghandler(self):
+        from ZConfig.components.logger import loghandler
+        host, selector = self.section.url
+        return loghandler.HTTPHandler(host, selector, self.section.method)
+
+class SMTPHandlerFactory(HandlerFactory):
+    def create_loghandler(self):
+        from ZConfig.components.logger import loghandler
+        host, port = self.section.smtp_server
+        if not port:
+            mailhost = host
+        else:
+            mailhost = host, port
+        return loghandler.SMTPHandler(mailhost,
+                                      self.section.fromaddr,
+                                      self.section.toaddrs,
+                                      self.section.subject)

trunk/src/ZConfig/components/logger/handlers.xml

+<component prefix="ZConfig.components.logger.handlers">
+  <description>
+  </description>
+
+  <import package="ZConfig.components.logger" file="abstract.xml"/>
+
+  <sectiontype name="ZConfig.logger.base-log-handler">
+    <description>
+      Base type for most log handlers.  This is cannot be used as a
+      loghandler directly since it doesn't implement the loghandler
+      abstract section type.
+    </description>
+    <key name="dateformat"
+         default="%Y-%m-%dT%H:%M:%S"/>
+    <key name="level"
+         default="notset"
+         datatype="ZConfig.components.logger.datatypes.logging_level"/>
+  </sectiontype>
+
+  <sectiontype name="logfile"
+               datatype=".FileHandlerFactory"
+               implements="ZConfig.logger.handler"
+               extends="ZConfig.logger.base-log-handler">
+    <key name="path" required="yes"/>
+    <key name="format"
+         default="------\n%(asctime)s %(levelname)s %(name)s %(message)s"
+         datatype=".log_format"/>
+  </sectiontype>
+
+  <sectiontype name="syslog"
+               datatype=".SyslogHandlerFactory"
+               implements="ZConfig.logger.handler"
+               extends="ZConfig.logger.base-log-handler">
+    <key name="facility" default="user" datatype=".syslog_facility"/>
+    <key name="address" datatype="socket-address" default="localhost:514"/>
+    <key name="format"
+         default="%(name)s %(message)s"
+         datatype=".log_format"/>
+  </sectiontype>
+
+  <sectiontype name="win32-eventlog"
+               datatype=".Win32EventLogFactory"
+               implements="ZConfig.logger.handler"
+               extends="ZConfig.logger.base-log-handler">
+    <key name="appname" default="Zope"/>
+    <key name="format"
+         default="%(levelname)s %(name)s %(message)s"
+         datatype=".log_format"/>
+  </sectiontype>
+
+  <sectiontype name="http-logger"
+               datatype=".HTTPHandlerFactory"
+               implements="ZConfig.logger.handler"
+               extends="ZConfig.logger.base-log-handler">
+    <key name="url" default="http://localhost/" datatype=".http_handler_url"/>
+    <key name="method" default="GET" datatype=".get_or_post"/>
+    <key name="format"
+         default="%(asctime)s %(levelname)s %(name)s %(message)s"
+         datatype=".log_format"/>
+  </sectiontype>
+
+  <sectiontype name="email-notifier"
+               datatype=".SMTPHandlerFactory"
+               implements="ZConfig.logger.handler"
+               extends="ZConfig.logger.base-log-handler">
+    <key name="from" required="yes" attribute="fromaddr"/>
+    <multikey name="to" required="yes" attribute="toaddrs"/>
+    <key name="subject" default="Message from Zope"/>
+    <key name="smtp-server" default="localhost" datatype="inet-address"/>
+    <key name="format"
+         default="%(asctime)s %(levelname)s %(name)s %(message)s"
+         datatype=".log_format"/>
+  </sectiontype>
+
+</component>

trunk/src/ZConfig/components/logger/logger.py

+##############################################################################
+#
+# Copyright (c) 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""ZConfig factory datatypes for loggers."""
+
+from ZConfig.components.logger.factory import Factory
+
+
+class LoggerFactoryBase(Factory):
+    """Base class for logger factories.
+
+    Factory used to create loggers while delaying actual logger
+    instance construction.  We need to do this because we may want to
+    reference a logger before actually instantiating it (for example,
+    to allow the app time to set an effective user).  An instance of
+    this wrapper is a callable which, when called, returns a logger
+    object.
+    """
+
+    def __init__(self, section):
+        Factory.__init__(self)
+        self.level = section.level
+        self.handler_factories = section.handlers
+
+    def create(self):
+        # set the logger up
+        import logging
+        logger = logging.getLogger(self.name)
+        logger.setLevel(self.level)
+        if self.handler_factories:
+            for handler_factory in self.handler_factories:
+                handler = handler_factory()
+                logger.addHandler(handler)
+        else:
+            from ZConfig.components.logger import loghandler
+            logger.addHandler(loghandler.NullHandler())
+        return logger
+
+    def startup(self):
+        # make sure we've instantiated the logger
+        self()
+
+    def getLowestHandlerLevel(self):
+        """Return the lowest log level provided by any configured handler.
+
+        If all handlers and the logger itself have level==NOTSET, this
+        returns NOTSET.
+        """
+        import logging
+        lowest = self.level
+        for factory in self.handler_factories:
+            level = factory.getLevel()
+            if level != logging.NOTSET:
+                if lowest == logging.NOTSET:
+                    lowest = level
+                else:
+                    lowest = min(lowest, level)
+        return lowest
+
+    def reopen(self):
+        """Re-open any handlers for which this is a meaningful operation.
+
+        This only works on handlers on the logger provided by this
+        factory directly; handlers for child loggers are not affected.
+        (This can be considered a bug, but is sufficient at the
+        moment.)
+        """
+        logger = self()
+        for handler in logger.handlers:
+            reopen = getattr(handler, "reopen", None)
+            if reopen is not None and callable(reopen):
+                reopen()
+
+
+class EventLogFactory(LoggerFactoryBase):
+    """Logger factory that returns the root logger."""
+
+    name = None
+
+
+class LoggerFactory(LoggerFactoryBase):
+    """Logger factory that returns the named logger."""
+
+    def __init__(self, section):
+        LoggerFactoryBase.__init__(self, section)
+        self.name = section.name
+        self.propagate = section.propagate
+
+    def create(self):
+        logger = LoggerFactoryBase.create(self)
+        logger.propagate = self.propagate
+        return logger

trunk/src/ZConfig/components/logger/logger.xml

+<component prefix="ZConfig.components.logger.logger">
+  <description>
+  </description>
+
+  <import package="ZConfig.components.logger" file="abstract.xml"/>
+  <import package="ZConfig.components.logger" file="base-logger.xml"/>
+  <import package="ZConfig.components.logger" file="eventlog.xml"/>
+
+  <sectiontype name="logger"
+               datatype=".LoggerFactory"
+               extends="ZConfig.logger.base-logger"
+               implements="ZConfig.logger.log">
+    <key name="propagate"
+         datatype="boolean"
+         default="true">
+      <description>
+        Indicates whether events that reach this logger should be
+        propogated toward the root of the logger hierarchy.  If true
+        (the default), events will be passed to the logger's parent
+        after being handled.  If false, events will be handled and the
+        parent will not be informed.  There is not a way to control
+        propogation by the severity of the event.
+      </description>
+    </key>
+
+    <key name="name"
+         datatype="dotted-name"
+         required="yes">
+      <description>
+        The dotted name of the logger.  This give it a location in the
+        logging hierarchy.  Most applications provide a specific set
+        of subsystem names for which logging is meaning; consult the
+        application documentation for the set of names that are
+        actually interesting for the application.
+      </description>
+    </key>
+  </sectiontype>
+
+</component>

trunk/src/ZConfig/components/logger/loghandler.py

+##############################################################################
+#
+# Copyright (c) 2001 Zope Corporation and Contributors. All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+"""Handlers which can plug into a PEP 282 logger."""
+
+import os.path
+import sys
+
+from logging import Handler, StreamHandler
+from logging.handlers import SysLogHandler, BufferingHandler
+from logging.handlers import HTTPHandler, SMTPHandler
+from logging.handlers import NTEventLogHandler as Win32EventLogHandler
+
+
+class FileHandler(StreamHandler):
+    """File handler which supports reopening of logs.
+
+    Re-opening should be used instead of the 'rollover' feature of
+    the FileHandler from the standard library's logging package.
+    """
+
+    def __init__(self, filename, mode="a"):
+        filename = os.path.abspath(filename)
+        StreamHandler.__init__(self, open(filename, mode))
+        self.baseFilename = filename
+        self.mode = mode
+
+    def close(self):
+        self.stream.close()
+
+    def reopen(self):
+        self.close()
+        self.stream = open(self.baseFilename, self.mode)
+
+
+class NullHandler(Handler):
+    """Handler that does nothing."""
+
+    def emit(self, record):
+        pass
+
+    def handle(self, record):
+        pass
+
+
+class StartupHandler(BufferingHandler):
+    """Handler which stores messages in a buffer until later.
+
+    This is useful at startup before we can know that we can safely
+    write to a configuration-specified handler.
+    """
+
+    def __init__(self):
+        BufferingHandler.__init__(self, sys.maxint)
+
+    def shouldFlush(self, record):
+        return False
+
+    def flushBufferTo(self, target):
+        while self.buffer:
+            target.handle(self.buffer.pop(0))

trunk/src/ZConfig/components/logger/tests/__init__.py

+# This is a Python package.

trunk/src/ZConfig/components/logger/tests/test_logger.py

+##############################################################################
+#
+# Copyright (c) 2002 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+"""Tests for logging configuration via ZConfig."""
+
+import cStringIO as StringIO
+import logging
+import sys
+import tempfile
+import unittest
+
+import ZConfig
+
+from ZConfig.components.logger import datatypes
+from ZConfig.components.logger import handlers
+from ZConfig.components.logger import loghandler
+
+
+class LoggingTestBase(unittest.TestCase):
+
+    # XXX This tries to save and restore the state of logging around
+    # the test.  Somewhat surgical; there may be a better way.
+
+    def setUp(self):
+        self._old_logger = logging.getLogger()
+        self._old_level = self._old_logger.level
+        self._old_handlers = self._old_logger.handlers[:]
+        self._old_logger.handlers[:] = []
+        self._old_logger.setLevel(logging.WARN)
+
+    def tearDown(self):
+        for h in self._old_logger.handlers:
+            self._old_logger.removeHandler(h)
+        for h in self._old_handlers:
+            self._old_logger.addHandler(h)
+        self._old_logger.setLevel(self._old_level)
+
+
+class TestConfig(LoggingTestBase):
+
+    _schema = None
+    _schematext = """
+      <schema>
+        <import package='ZConfig.components.logger'/>
+        <section type='eventlog' name='*' attribute='eventlog'/>
+      </schema>
+    """
+
+    def get_schema(self):
+        if self._schema is None:
+            sio = StringIO.StringIO(self._schematext)
+            self.__class__._schema = ZConfig.loadSchemaFile(sio)
+        return self._schema
+
+    def get_config(self, text):
+        conf, handler = ZConfig.loadConfigFile(self.get_schema(),
+                                               StringIO.StringIO(text))
+        self.assert_(not handler)
+        return conf
+
+    def test_logging_level(self):
+        # Make sure the expected names are supported; it's not clear
+        # how to check the values in a meaningful way.
+        # Just make sure they're case-insensitive.
+        convert = datatypes.logging_level
+        for name in ["notset", "all", "trace", "debug", "blather",
+                     "info", "warn", "warning", "error", "fatal",
+                     "critical"]:
+            self.assertEqual(convert(name), convert(name.upper()))
+        self.assertRaises(ValueError, convert, "hopefully-not-a-valid-value")
+
+    def test_http_method(self):
+        convert = handlers.get_or_post
+        self.assertEqual(convert("get"), "GET")
+        self.assertEqual(convert("GET"), "GET")
+        self.assertEqual(convert("post"), "POST")
+        self.assertEqual(convert("POST"), "POST")
+        self.assertRaises(ValueError, convert, "")
+        self.assertRaises(ValueError, convert, "foo")
+
+    def test_syslog_facility(self):
+        convert = handlers.syslog_facility
+        for name in ["auth", "authpriv", "cron", "daemon", "kern",
+                     "lpr", "mail", "news", "security", "syslog",
+                     "user", "uucp", "local0", "local1", "local2",
+                     "local3", "local4", "local5", "local6", "local7"]:
+            self.assertEqual(convert(name), name)
+            self.assertEqual(convert(name.upper()), name)
+        self.assertRaises(ValueError, convert, "hopefully-never-a-valid-value")
+
+    def test_config_without_logger(self):
+        conf = self.get_config("")
+        self.assert_(conf.eventlog is None)
+
+    def test_config_without_handlers(self):
+        logger = self.check_simple_logger("<eventlog/>")
+        # Make sure there's a NullHandler, since a warning gets
+        # printed if there are no handlers:
+        self.assertEqual(len(logger.handlers), 1)
+        self.assert_(isinstance(logger.handlers[0],
+                                loghandler.NullHandler))
+
+    def test_with_logfile(self):
+        import os
+        fn = tempfile.mktemp()
+        logger = self.check_simple_logger("<eventlog>\n"
+                                          "  <logfile>\n"
+                                          "    path %s\n"
+                                          "    level debug\n"
+                                          "  </logfile>\n"
+                                          "</eventlog>" % fn)
+        logfile = logger.handlers[0]
+        self.assertEqual(logfile.level, logging.DEBUG)
+        self.assert_(isinstance(logfile, loghandler.FileHandler))
+        logfile.close()
+        os.remove(fn)
+
+    def test_with_stderr(self):
+        self.check_standard_stream("stderr")
+
+    def test_with_stdout(self):
+        self.check_standard_stream("stdout")
+
+    def check_standard_stream(self, name):
+        old_stream = getattr(sys, name)
+        conf = self.get_config("""
+            <eventlog>
+              <logfile>
+                level info
+                path %s
+              </logfile>
+            </eventlog>
+            """ % name.upper())
+        self.assert_(conf.eventlog is not None)
+        # The factory has already been created; make sure it picks up
+        # the stderr we set here when we create the logger and
+        # handlers:
+        sio = StringIO.StringIO()
+        setattr(sys, name, sio)
+        try:
+            logger = conf.eventlog()
+        finally:
+            setattr(sys, name, old_stream)
+        logger.warn("woohoo!")
+        self.assert_(sio.getvalue().find("woohoo!") >= 0)
+
+    def test_with_syslog(self):
+        logger = self.check_simple_logger("<eventlog>\n"
+                                          "  <syslog>\n"
+                                          "    level error\n"
+                                          "    facility local3\n"
+                                          "  </syslog>\n"
+                                          "</eventlog>")
+        syslog = logger.handlers[0]
+        self.assertEqual(syslog.level, logging.ERROR)
+        self.assert_(isinstance(syslog, loghandler.SysLogHandler))
+
+    def test_with_http_logger_localhost(self):
+        logger = self.check_simple_logger("<eventlog>\n"
+                                          "  <http-logger>\n"
+                                          "    level error\n"
+                                          "    method post\n"
+                                          "  </http-logger>\n"
+                                          "</eventlog>")
+        handler = logger.handlers[0]
+        self.assertEqual(handler.host, "localhost")
+        # XXX The "url" attribute of the handler is misnamed; it
+        # really means just the selector portion of the URL.
+        self.assertEqual(handler.url, "/")
+        self.assertEqual(handler.level, logging.ERROR)
+        self.assertEqual(handler.method, "POST")
+        self.assert_(isinstance(handler, loghandler.HTTPHandler))
+
+    def test_with_http_logger_remote_host(self):
+        logger = self.check_simple_logger("<eventlog>\n"
+                                          "  <http-logger>\n"
+                                          "    method get\n"
+                                          "    url http://example.com/log/\n"
+                                          "  </http-logger>\n"
+                                          "</eventlog>")
+        handler = logger.handlers[0]
+        self.assertEqual(handler.host, "example.com")
+        # XXX The "url" attribute of the handler is misnamed; it
+        # really means just the selector portion of the URL.
+        self.assertEqual(handler.url, "/log/")
+        self.assertEqual(handler.level, logging.NOTSET)
+        self.assertEqual(handler.method, "GET")
+        self.assert_(isinstance(handler, loghandler.HTTPHandler))
+
+    def test_with_email_notifier(self):
+        logger = self.check_simple_logger("<eventlog>\n"
+                                          "  <email-notifier>\n"
+                                          "    to sysadmin@example.com\n"
+                                          "    to sa-pager@example.com\n"
+                                          "    from zlog-user@example.com\n"
+                                          "    level fatal\n"
+                                          "  </email-notifier>\n"
+                                          "</eventlog>")
+        handler = logger.handlers[0]
+        self.assertEqual(handler.toaddrs, ["sysadmin@example.com",
+                                           "sa-pager@example.com"])
+        self.assertEqual(handler.fromaddr, "zlog-user@example.com")
+        self.assertEqual(handler.level, logging.FATAL)
+
+    def check_simple_logger(self, text, level=logging.INFO):
+        conf = self.get_config(text)
+        self.assert_(conf.eventlog is not None)
+        self.assertEqual(conf.eventlog.level, level)
+        logger = conf.eventlog()
+        self.assert_(isinstance(logger, logging.Logger))
+        self.assertEqual(len(logger.handlers), 1)
+        return logger
+
+
+def test_suite():
+    return unittest.makeSuite(TestConfig)
+
+if __name__ == '__main__':
+    unittest.main(defaultTest="test_suite")

trunk/src/ZConfig/datatypes.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Selection of standard datatypes for ZConfig."""
+
+import os
+import re
+import sys
+import datetime
+
+# types.StringTypes was added in Python 2.2; basestring in 2.3
+try:
+    unicode
+except NameError:
+    have_unicode = False
+else:
+    have_unicode = True
+
+
+class MemoizedConversion:
+    """Conversion helper that caches the results of expensive conversions."""
+
+    def __init__(self, conversion):
+        self._memo = {}
+        self._conversion = conversion
+
+    def __call__(self, value):
+        try:
+            return self._memo[value]
+        except KeyError:
+            v = self._conversion(value)
+            self._memo[value] = v
+            return v
+
+
+class RangeCheckedConversion:
+    """Conversion helper that range checks another conversion."""
+
+    def __init__(self, conversion, min=None, max=None):
+        self._min = min
+        self._max = max
+        self._conversion = conversion
+
+    def __call__(self, value):
+        v = self._conversion(value)
+        if self._min is not None and v < self._min:
+            raise ValueError("%s is below lower bound (%s)"
+                             % (`v`, `self._min`))
+        if self._max is not None and v > self._max:
+            raise ValueError("%s is above upper bound (%s)"
+                             % (`v`, `self._max`))
+        return v
+
+
+class RegularExpressionConversion:
+    reason = "value did not match regular expression"
+
+    def __init__(self, regex):
+        self._rx = re.compile(regex)
+
+    def __call__(self, value):
+        m = self._rx.match(value)
+        if m and m.group() == value:
+            return value
+        else:
+            raise ValueError("%s: %s" % (self.reason, repr(value)))
+
+
+def check_locale(value):
+    import locale
+    prev = locale.setlocale(locale.LC_ALL)
+    try:
+        try:
+            locale.setlocale(locale.LC_ALL, value)
+        finally:
+            locale.setlocale(locale.LC_ALL, prev)
+    except locale.Error:
+        raise ValueError(
+            'The specified locale "%s" is not supported by your system.\n'
+            'See your operating system documentation for more\n'
+            'information on locale support.' % value)
+    else:
+        return value
+
+
+class BasicKeyConversion(RegularExpressionConversion):
+    def __init__(self):
+        RegularExpressionConversion.__init__(self, "[a-zA-Z][-._a-zA-Z0-9]*")
+
+    def __call__(self, value):
+        value = str(value)
+        return RegularExpressionConversion.__call__(self, value).lower()
+
+
+class ASCIIConversion(RegularExpressionConversion):
+    def __call__(self, value):
+        value = RegularExpressionConversion.__call__(self, value)
+        if have_unicode and isinstance(value, unicode):
+            value = value.encode("ascii")
+        return value
+
+
+_ident_re = "[_a-zA-Z][_a-zA-Z0-9]*"
+
+class IdentifierConversion(ASCIIConversion):
+    reason = "not a valid Python identifier"
+
+    def __init__(self):
+        ASCIIConversion.__init__(self, _ident_re)
+
+
+class DottedNameConversion(ASCIIConversion):
+    reason = "not a valid dotted name"
+
+    def __init__(self):
+        ASCIIConversion.__init__(self,
+                                 r"%s(?:\.%s)*" % (_ident_re, _ident_re))
+
+
+class DottedNameSuffixConversion(ASCIIConversion):
+    reason = "not a valid dotted name or suffix"
+
+    def __init__(self):
+        ASCIIConversion.__init__(self,
+                                 r"(?:%s)(?:\.%s)*|(?:\.%s)+"
+                                 % (_ident_re, _ident_re, _ident_re))
+
+
+def integer(value):
+    try:
+        return int(value)
+    except ValueError:
+        return long(value)
+    except OverflowError:
+        return long(value)
+
+
+def null_conversion(value):
+    return value
+
+
+def asBoolean(s):
+    """Convert a string value to a boolean value."""
+    ss = str(s).lower()
+    if ss in ('yes', 'true', 'on'):
+        return True
+    elif ss in ('no', 'false', 'off'):
+        return False
+    else:
+        raise ValueError("not a valid boolean value: " + repr(s))
+
+
+def string_list(s):
+    """Convert a string to a list of strings using .split()."""
+    return s.split()
+
+
+port_number = RangeCheckedConversion(integer, min=1, max=0xffff).__call__
+
+
+if sys.platform[:3] == "win":
+    DEFAULT_HOST = "localhost"
+else:
+    DEFAULT_HOST = ""
+
+def inet_address(s):
+    # returns (host, port) tuple
+    host = ''
+    port = None
+    if ":" in s:
+        host, s = s.split(":", 1)
+        if s:
+            port = port_number(s)
+        host = host.lower()
+    else:
+        try:
+            port = port_number(s)
+        except ValueError:
+            if len(s.split()) != 1:
+                raise ValueError("not a valid host name: " + repr(s))
+            host = s.lower()
+    if not host:
+        host = DEFAULT_HOST
+    return host, port
+
+
+class SocketAddress:
+    def __init__(self, s):
+        # returns (family, address) tuple
+        import socket
+        if "/" in s or s.find(os.sep) >= 0:
+            self.family = getattr(socket, "AF_UNIX", None)
+            self.address = s
+        else:
+            self.family = socket.AF_INET
+            self.address = inet_address(s)
+
+def float_conversion(v):
+    if isinstance(v, basestring):
+        if v.lower() in ["inf", "-inf", "nan"]:
+            raise ValueError(`v` + " is not a portable float representation")
+    return float(v)
+
+class IpaddrOrHostname(RegularExpressionConversion):
+    def __init__(self):
+        # IP address regex from the Perl Cookbook, Recipe 6.23 (revised ed.)
+        # We allow underscores in hostnames although this is considered
+        # illegal according to RFC1034.
+        expr = (r"(^(\d|[01]?\d\d|2[0-4]\d|25[0-5])\." #ipaddr
+                r"(\d|[01]?\d\d|2[0-4]\d|25[0-5])\." #ipaddr cont'd
+                r"(\d|[01]?\d\d|2[0-4]\d|25[0-5])\." #ipaddr cont'd
+                r"(\d|[01]?\d\d|2[0-4]\d|25[0-5])$)" #ipaddr cont'd
+                r"|([A-Za-z_][-A-Za-z0-9_.]*[-A-Za-z0-9_])") # or hostname
+        RegularExpressionConversion.__init__(self, expr)
+
+    def __call__(self, value):
+        return RegularExpressionConversion.__call__(self, value).lower()
+
+def existing_directory(v):
+    nv = os.path.expanduser(v)
+    if os.path.isdir(nv):
+        return nv
+    raise ValueError, '%s is not an existing directory' % v
+
+def existing_path(v):
+    nv = os.path.expanduser(v)
+    if os.path.exists(nv):
+        return nv
+    raise ValueError, '%s is not an existing path' % v
+
+def existing_file(v):
+    nv = os.path.expanduser(v)
+    if os.path.exists(nv):
+        return nv
+    raise ValueError, '%s is not an existing file' % v
+
+def existing_dirpath(v):
+    nv = os.path.expanduser(v)
+    dir = os.path.dirname(nv)
+    if not dir:
+        # relative pathname with no directory component
+        return nv
+    if os.path.isdir(dir):
+        return nv
+    raise ValueError, ('The directory named as part of the path %s '
+                       'does not exist.' % v)
+
+
+class SuffixMultiplier:
+    # d is a dictionary of suffixes to integer multipliers.  If no suffixes
+    # match, default is the multiplier.  Matches are case insensitive.  Return
+    # values are in the fundamental unit.
+    def __init__(self, d, default=1):
+        self._d = d
+        self._default = default
+        # all keys must be the same size
+        self._keysz = None
+        for k in d.keys():
+            if self._keysz is None:
+                self._keysz = len(k)
+            else:
+                assert self._keysz == len(k)
+
+    def __call__(self, v):
+        v = v.lower()
+        for s, m in self._d.items():
+            if v[-self._keysz:] == s:
+                return int(v[:-self._keysz]) * m
+        return int(v) * self._default
+
+
+def timedelta(s):
+    # Unlike the standard time-interval data type, which returns a float
+    # number of seconds, this datatype takes a wider range of syntax and
+    # returns a datetime.timedelta
+    #
+    # Accepts suffixes:
+    #    w - weeks
+    #    d - days
+    #    h - hours
+    #    m - minutes
+    #    s - seconds
+    #
+    # and all arguments may be integers or floats, positive or negative.
+    # More than one time interval suffix value may appear on the line, but
+    # they should all be separated by spaces, e.g.:
+    #
+    # sleep_time 4w 2d 7h 12m 0.00001s
+    weeks = days = hours = minutes = seconds = 0
+    for part in s.split():
+        val = float(part[:-1])
+        suffix = part[-1]
+        if suffix == 'w':
+            weeks = val
+        elif suffix == 'd':
+            days = val
+        elif suffix == 'h':
+            hours = val
+        elif suffix == 'm':
+            minutes = val
+        elif suffix == 's':
+            seconds = val
+        else:
+            raise TypeError('bad part %s in %s' % (part, s))
+    return datetime.timedelta(weeks=weeks, days=days, hours=hours,
+                              minutes=minutes, seconds=seconds)
+
+
+stock_datatypes = {
+    "boolean":           asBoolean,
+    "dotted-name":       DottedNameConversion(),
+    "dotted-suffix":     DottedNameSuffixConversion(),
+    "identifier":        IdentifierConversion(),
+    "integer":           integer,
+    "float":             float_conversion,
+    "string":            str,
+    "string-list":       string_list,
+    "null":              null_conversion,
+    "locale":            MemoizedConversion(check_locale),
+    "port-number":       port_number,
+    "basic-key":         BasicKeyConversion(),
+    "inet-address":      inet_address,
+    "socket-address":    SocketAddress,
+    "ipaddr-or-hostname":IpaddrOrHostname(),
+    "existing-directory":existing_directory,
+    "existing-path":     existing_path,
+    "existing-file":     existing_file,
+    "existing-dirpath":  existing_dirpath,
+    "byte-size":         SuffixMultiplier({'kb': 1024,
+                                           'mb': 1024*1024,
+                                           'gb': 1024*1024*1024L,
+                                           }),
+    "time-interval":     SuffixMultiplier({'s': 1,
+                                           'm': 60,
+                                           'h': 60*60,
+                                           'd': 60*60*24,
+                                           }),
+    "timedelta":         timedelta,
+    }
+
+
+class Registry:
+    def __init__(self, stock=None):
+        if stock is None:
+            stock = stock_datatypes.copy()
+        self._stock = stock
+        self._other = {}
+        self._basic_key = None
+
+    def get(self, name):
+        if '.' not in name:
+            if self._basic_key is None:
+                self._basic_key = self._other.get("basic-key")
+                if self._basic_key is None:
+                    self._basic_key = self._stock.get("basic-key")
+                if self._basic_key is None:
+                    self._basic_key = stock_datatypes["basic-key"]
+            name = self._basic_key(name)
+        t = self._stock.get(name)
+        if t is None:
+            t = self._other.get(name)
+            if t is None:
+                t = self.search(name)
+        return t
+
+    def register(self, name, conversion):
+        if self._stock.has_key(name):
+            raise ValueError("datatype name conflicts with built-in type: "
+                             + `name`)
+        if self._other.has_key(name):
+            raise ValueError("datatype name already registered: " + `name`)
+        self._other[name] = conversion
+
+    def search(self, name):
+        if not "." in name:
+            raise ValueError("unloadable datatype name: " + `name`)
+        components = name.split('.')
+        start = components[0]
+        g = {}
+        package = __import__(start, g, g)
+        modulenames = [start]
+        for component in components[1:]:
+            modulenames.append(component)
+            try:
+                package = getattr(package, component)
+            except AttributeError:
+                n = '.'.join(modulenames)
+                package = __import__(n, g, g, component)
+        self._other[name] = package
+        return package

trunk/src/ZConfig/doc/Makefile

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+# Rules to convert the documentation to a single PDF file.
+#
+# PostScript, HTML, and plain text output are also supported, though
+# PDF is the default.
+#
+# See the README.txt file for information on the mkhowto program used
+# to generate the formatted versions of the documentation.
+
+.PHONY:	default all html pdf ps text
+
+default:  pdf
+all:	  html pdf ps text
+
+html:   zconfig/zconfig.html
+pdf:	zconfig.pdf
+ps:	zconfig.ps
+text:   zconfig.txt
+
+zconfig/zconfig.html:  zconfig.tex schema.dtd xmlmarkup.perl
+	mkhowto --html $<
+
+zconfig.pdf:  zconfig.tex schema.dtd xmlmarkup.sty
+	mkhowto --pdf $<
+
+zconfig.ps:  zconfig.tex schema.dtd xmlmarkup.sty
+	mkhowto --postscript $<
+
+zconfig.txt: zconfig.tex schema.dtd xmlmarkup.sty
+	mkhowto --text $<
+
+clean:
+	rm -f zconfig.l2h zconfig.l2h~
+
+clobber:  clean
+	rm -f zconfig.pdf zconfig.ps zconfig.txt
+	rm -rf zconfig

trunk/src/ZConfig/doc/README.txt

+The zconfig.tex document in this directory contains the reference
+documentation for the ZConfig package.  This documentation is written
+using the Python LaTeX styles.
+
+To format the documentation, get a copy of the Python documentation
+tools (the Doc/ directory from the Python sources), and create a
+symlink to the tools/mkhowto script from some convenient bin/
+directory.  You will need to have a fairly complete set of
+documentation tools installed on your platform; see
+
+    http://www.python.org/doc/current/doc/doc.html
+
+for more information on the tools.
+
+This documentation requires the latest version of the Python
+documentation tools from CVS.

trunk/src/ZConfig/doc/schema.dtd

+<!--
+  *************************************************************************
+  Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+  All Rights Reserved.
+
+  This software is subject to the provisions of the Zope Public License,
+  Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+  THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+  WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+  WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+  FOR A PARTICULAR PURPOSE.
+  *************************************************************************
+
+  Please note that not all documents that conform to this DTD are
+  legal ZConfig schema.  The ZConfig reference manual describes many
+  constraints that are important to understanding ZConfig schema.
+  -->
+
+<!-- DTD for ZConfig schema documents. -->
+
+<!ELEMENT schema (description?, metadefault?, example?,
+                  import*,
+                  (sectiontype | abstracttype)*,
+                  (section | key | multisection | multikey)*)>
+<!ATTLIST schema
+          extends    NMTOKEN  #IMPLIED
+          prefix     NMTOKEN  #IMPLIED
+          handler    NMTOKEN  #IMPLIED
+          keytype    NMTOKEN  #IMPLIED
+          datatype   NMTOKEN  #IMPLIED>
+
+<!ELEMENT component (description?, (sectiontype | abstracttype)*)>
+<!ATTLIST component
+          prefix     NMTOKEN  #IMPLIED>
+
+<!ELEMENT import EMPTY>
+<!ATTLIST import
+          file       CDATA    #IMPLIED
+          package    NMTOKEN  #IMPLIED
+          src        CDATA    #IMPLIED>
+
+<!ELEMENT description (#PCDATA)*>
+<!ATTLIST description
+          format     NMTOKEN  #IMPLIED>
+
+<!ELEMENT metadefault (#PCDATA)*>
+<!ELEMENT example     (#PCDATA)*>
+
+<!ELEMENT sectiontype (description?, 
+                       (section | key | multisection | multikey)*)>
+<!ATTLIST sectiontype
+          name       NMTOKEN  #REQUIRED
+          prefix     NMTOKEN  #IMPLIED
+          keytype    NMTOKEN  #IMPLIED
+          datatype   NMTOKEN  #IMPLIED
+          implements NMTOKEN  #IMPLIED
+          extends    NMTOKEN  #IMPLIED>
+
+<!ELEMENT abstracttype (description?)>
+<!ATTLIST abstracttype
+          name       NMTOKEN  #REQUIRED
+          prefix     NMTOKEN  #IMPLIED>
+
+<!ELEMENT default    (#PCDATA)*>
+<!ATTLIST default
+          key        CDATA    #IMPLIED>
+
+<!ELEMENT key (description?, metadefault?, example?, default*)>
+<!ATTLIST key
+          name       CDATA    #REQUIRED
+          attribute  NMTOKEN  #IMPLIED
+          datatype   NMTOKEN  #IMPLIED
+          handler    NMTOKEN  #IMPLIED
+          required   (yes|no) "no"
+          default    CDATA    #IMPLIED>
+
+<!ELEMENT multikey (description?, metadefault?, example?, default*)>
+<!ATTLIST multikey
+          name       CDATA    #REQUIRED
+          attribute  NMTOKEN  #IMPLIED
+          datatype   NMTOKEN  #IMPLIED
+          handler    NMTOKEN  #IMPLIED
+          required   (yes|no) "no">
+
+<!ELEMENT section (description?)>
+<!ATTLIST section
+          name       CDATA    #REQUIRED
+          attribute  NMTOKEN  #IMPLIED
+          type       NMTOKEN  #REQUIRED
+          handler    NMTOKEN  #IMPLIED
+          required   (yes|no) "no">
+
+<!ELEMENT multisection (description?)>
+<!ATTLIST multisection
+          name       CDATA    #REQUIRED
+          attribute  NMTOKEN  #IMPLIED
+          type       NMTOKEN  #REQUIRED
+          handler    NMTOKEN  #IMPLIED
+          required   (yes|no) "no">

trunk/src/ZConfig/doc/xmlmarkup.perl

+##############################################################################
+#
+# Copyright (c) 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+# LaTeX2HTML support for the xmlmarkup package.  Doesn't do indexing.
+
+package main;
+
+
+sub do_cmd_element{
+    local($_) = @_;
+    my $name = next_argument();
+    return "<tt class='element'>$name</tt>" . $_;
+}
+
+sub do_cmd_attribute{
+    local($_) = @_;
+    my $name = next_argument();
+    return "<tt class='attribute'>$name</tt>" . $_;
+}
+
+sub do_env_attributedesc{
+    local($_) = @_;
+    my $name = next_argument();
+    my $valuetype = next_argument();
+    return ("\n<dl class='macrodesc'>"
+            . "\n<dt><b><tt class='macro'>$name</tt></b>"
+            . "&nbsp;&nbsp;&nbsp;($valuetype)"
+            . "\n<dd>"
+            . $_
+            . "</dl>");
+}
+
+sub do_env_elementdesc{
+    local($_) = @_;
+    my $name = next_argument();
+    my $contentmodel = next_argument();
+    return ("\n<dl class='elementdesc'>"
+            . "\n<dt class='start-tag'><tt>&lt;"
+            . "<b class='element'>$name</b>&gt;</tt>"
+            . "\n<dd class='content-model'>$contentmodel"
+            . "\n<dt class='endtag'><tt>&lt;/"
+            . "<b class='element'>$name</b>&gt;</tt>"
+            . "\n<dd class='descrition'>"
+            . $_
+            . "</dl>");
+}
+
+1;				# Must end with this, because Perl is bogus.

trunk/src/ZConfig/doc/xmlmarkup.sty

+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%
+% Copyright (c) 2003 Zope Corporation and Contributors.
+% All Rights Reserved.
+%
+% This software is subject to the provisions of the Zope Public License,
+% Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+% THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+% WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+% WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+% FOR A PARTICULAR PURPOSE.
+%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+% Define some simple markup for the LaTeX command documentation:
+
+\ProvidesPackage{xmlmarkup}
+\RequirePackage{python}      % fulllineitems environment
+
+\newcommand{\element}[1]{\code{#1}}
+\newcommand{\attribute}[1]{\code{#1}}
+
+% \begin{elementdesc}{type}{content-model}
+\newenvironment{elementdesc}[2]{
+  \begin{fulllineitems}
+    \item[\code{\textless{\bfseries #1}\textgreater}]
+    \code{#2}
+    \item[\code{\textless/{\bfseries #1}\textgreater}]
+    \index{#1 element@\py@idxcode{#1} element}
+    \index{elements!#1@\py@idxcode{#1}}
+}{\end{fulllineitems}}
+
+% \begin{attributedesc}{name}{content-type}
+\newenvironment{attributedesc}[2]{
+  \begin{fulllineitems}
+    \item[\code{\bfseries#1}{\quad(#2)}]
+    \index{#1@\py@idxcode{#1}}
+}{\end{fulllineitems}}

trunk/src/ZConfig/doc/zconfig.tex

+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%
+% Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+% All Rights Reserved.
+%
+% This software is subject to the provisions of the Zope Public License,
+% Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+% THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+% WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+% WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+% FOR A PARTICULAR PURPOSE.
+%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\documentclass{howto}
+\usepackage{xmlmarkup}
+
+\newcommand{\datatype}[1]{\strong{#1}}
+
+\title{ZConfig Package Reference}
+
+%\date{27 October 2003}
+\release{2.2}
+\setshortversion{2.2}
+
+\author{Zope Corporation}
+\authoraddress{
+    Lafayette Technology Center\\
+    513 Prince Edward Street\\
+    Fredericksburg, VA 22401\\
+    \url{http://www.zope.com/}
+}
+
+\begin{document}
+\maketitle
+
+\begin{abstract}
+\noindent
+This document describes the syntax and API used in configuration files
+for components of a Zope installation written by Zope Corporation.  This
+configuration mechanism is itself configured using a schema specification
+written in XML.
+\end{abstract}
+
+\tableofcontents
+
+
+\section{Introduction \label{intro}}
+
+Zope uses a common syntax and API for configuration files designed for
+software components written by Zope Corporation.  Third-party software
+which is also part of a Zope installation may use a different syntax,
+though any software is welcome to use the syntax used by Zope
+Corporation.  Any software written in Python is free to use the
+\module{ZConfig} software to load such configuration files in order to
+ensure compatibility.  This software is covered by the Zope Public
+License, version 2.0.
+
+The \module{ZConfig} package has been tested with Python 2.3.  Older
+versions of Python are not supported.
+\module{ZConfig} only relies on the Python standard library.
+
+Configurations which use \module{ZConfig} are described using
+\dfn{schema}.  A schema is a specification for the allowed structure
+and content of the configuration.  \module{ZConfig} schema are written
+using a small XML-based language.  The schema language allows the
+schema author to specify the names of the keys allowed at the top
+level and within sections, to define the types of sections which may
+be used (and where), the types of each values, whether a key or
+section must be specified or is optional, default values for keys, and
+whether a value can be given only once or repeatedly.
+
+
+\section{Configuration Syntax \label{syntax}}
+
+Like the \ulink{\module{ConfigParser}}
+{http://docs.python.org/lib/module-ConfigParser.html}
+format, this format supports key-value pairs arranged in sections.
+Unlike the \module{ConfigParser} format, sections are typed and can be
+organized hierarchically.
+Additional files may be included if needed.  Schema components not
+specified in the application schema can be imported from the
+configuration file.  Though both formats are substantially
+line-oriented, this format is more flexible.
+
+The intent of supporting nested section is to allow setting up the
+configurations for loosely-associated components in a container.  For
+example, each process running on a host might get its configuration
+section from that host's section of a shared configuration file.
+
+The top level of a configuration file consists of a series of
+inclusions, key-value pairs, and sections.
+
+Comments can be added on lines by themselves.  A comment has a
+\character{\#} as the first non-space character and extends to the end
+of the line:
+
+\begin{verbatim}
+# This is a comment
+\end{verbatim}
+
+An inclusion is expressed like this:
+
+\begin{verbatim}
+%include defaults.conf
+\end{verbatim}
+
+The resource to be included can be specified by a relative or absolute
+URL, resolved relative to the URL of the resource the
+\keyword{\%include} directive is located in.
+
+
+A key-value pair is expressed like this:
+
+\begin{verbatim}
+key value
+\end{verbatim}
+
+The key may include any non-white characters except for parentheses.
+The value contains all the characters between the key and the end of
+the line, with surrounding whitespace removed.
+
+Since comments must be on lines by themselves, the \character{\#}
+character can be part of a value:
+
+\begin{verbatim}
+key value # still part of the value
+\end{verbatim}
+
+Sections may be either empty or non-empty.  An empty section may be
+used to provide an alias for another section.
+
+A non-empty section starts with a header, contains configuration
+data on subsequent lines, and ends with a terminator.
+
+The header for a non-empty section has this form (square brackets
+denote optional parts):
+
+\begin{alltt}
+<\var{section-type} \optional{\var{name}} >
+\end{alltt}
+
+\var{section-type} and \var{name} all have the same syntactic
+constraints as key names.
+
+The terminator looks like this:
+
+\begin{alltt}
+</\var{section-type}>
+\end{alltt}
+
+The configuration data in a non-empty section consists of a sequence
+of one or more key-value pairs and sections.  For example:
+
+\begin{verbatim}
+<my-section>
+    key-1 value-1
+    key-2 value-2
+
+    <another-section>
+        key-3 value-3
+    </another-section>
+</my-section>
+\end{verbatim}
+
+(The indentation is used here for clarity, but is not required for
+syntactic correctness.)
+
+The header for empty sections is similar to that of non-empty
+sections, but there is no terminator:
+
+\begin{alltt}
+<\var{section-type} \optional{\var{name}} />
+\end{alltt}
+
+
+\subsection{Extending the Configuration Schema}
+
+As we'll see in section~\ref{writing-schema}, ``Writing Configuration
+Schema,'' what can be written in a configuration is controlled by
+schemas which can be built from \emph{components}.  These components
+can also be used to extend the set of implementations of objects the
+application can handle.  What this means when writing a configuration
+is that third-party implementations of application object types can be
+used wherever those application types are used in the configuration,
+if there's a \module{ZConfig} component available for that
+implementation.
+
+The configuration file can use an \keyword{\%import} directive to load
+a named component:
+
+\begin{verbatim}
+%import Products.Ape
+\end{verbatim}
+
+The text to the right of the \keyword{\%import} keyword must be the
+name of a Python package; the \module{ZConfig} component provided by
+that package will be loaded and incorporated into the schema being
+used to load the configuration file.  After the import, section types
+defined in the component may be used in the configuration.
+
+More detail is needed for this to really make sense.
+
+A schema may define section types which are \emph{abstract}; these
+cannot be used directly in a configuration, but multiple concrete
+section types can be defined which \emph{implement} the abstract
+types.  Wherever the application allows an abstract type to be used,
+any concrete type which implements that abstract type can be used in
+an actual configuration.
+
+The \keyword{\%import} directive allows loading schema components
+which provide alternate concrete section types which implement the
+abstract types defined by the application.  This allows third-party
+implementations of abstract types to be used in place of or in
+addition to implementations provided with the application.
+
+Consider an example application application which supports logging in
+the same way Zope 2 does.  There are some parameters which configure
+the general behavior of the logging mechanism, and an arbitrary number
+of \emph{log handlers} may be specified to control how the log
+messages are handled.  Several log handlers are provided by the
+application.  Here is an example logging configuration:
+
+\begin{verbatim}
+<eventlog>
+  level verbose
+
+  <logfile>
+    path /var/log/myapp/events.log
+  </logfile>
+</eventlog>
+\end{verbatim}
+
+A third-party component may provide a log handler to send
+high-priority alerts the system administrator's text pager or
+SMS-capable phone.  All that's needed is to install the implementation
+so it can be imported by Python, and modify the configuration:
+
+\begin{verbatim}
+%import my.pager.loghandler
+
+<eventlog>
+  level verbose
+
+  <logfile>
+    path /var/log/myapp/events.log
+  </logfile>
+
+  <pager>
+    number   1-800-555-1234
+    message  Something broke!
+  </pager>
+</eventlog>
+\end{verbatim}
+
+
+\subsection{Textual Substitution in Values}
+
+\module{ZConfig} provides a limited way to re-use portions of a value
+using simple string substitution.  To use this facility, define named
+bits of replacement text using the \keyword{\%define} directive, and
+reference these texts from values.
+
+The syntax for \keyword{\%define} is:
+
+\begin{alltt}
+%define \var{name} \optional{\var{value}}
+\end{alltt}
+
+The value of \var{name} must be a sequence of letters, digits, and
+underscores, and may not start with a digit; the namespace for these
+names is separate from the other namespaces used with
+\module{ZConfig}, and is case-insensitive.  If \var{value} is
+omitted, it will be the empty string.  If given, there must be
+whitespace between \var{name} and \var{value}; \var{value} will not
+include any whitespace on either side, just like values from key-value
+pairs.
+
+Names must be defined before they are used, and may not be
+re-defined.  All resources being parsed as part of a configuration
+share a single namespace for defined names.  This means that resources
+which may be included more than once should not define any names.
+
+References to defined names from configuration values use the syntax
+described for the \refmodule{ZConfig.substitution} module.
+Configuration values which include a \character{\$} as part of the
+actual value will need to use \code{\$\$} to get a single
+\character{\$} in the result.
+
+The values of defined names are processed in the same way as
+configuration values, and may contain references to named
+definitions.
+
+For example, the value for \code{key} will evaluate to \code{value}:
+
+\begin{verbatim}
+%define name value
+key $name
+\end{verbatim} %$ <-- bow to font-lock
+
+
+\section{Writing Configuration Schema \label{writing-schema}}
+
+\module{ZConfig} schema are written as XML documents.
+
+Data types are searched in a special namespace defined by the data
+type registry.  The default registry has slightly magical semantics:
+If the value can be matched to a standard data type when interpreted
+as a \datatype{basic-key}, the standard data type will be used.  If
+that fails, the value must be a \datatype{dotted-name} containing at
+least one dot, and a conversion function will be sought using the
+\method{search()} method of the data type registry used to load the
+schema.
+
+
+\subsection{Schema Elements \label{elements}}
+
+For each element, the content model is shown, followed by a
+description of how the element is used, and then a list of the
+available attributes.  For each attribute, the type of the value is
+given as either the name of a \module{ZConfig} datatype or an XML
+attribute value type.  Familiarity with XML's Document Type Definition
+language is helpful.
+
+The following elements are used to describe a schema:
+
+\begin{elementdesc}{schema}{description?, metadefault?, example?,
+                            import*,
+                            (sectiontype | abstracttype)*,
+                            (section | key | multisection |
+                            multikey)*}
+  Document element for a \module{ZConfig} schema.
+
+  \begin{attributedesc}{extends}{\datatype{space-separated-url-references}}
+  A list of URLs of base schemas from which this section type will inherit key,
+  section, and section type declarations.  If omitted, this schema
+  is defined using only the keys, sections, and section types contained within
+  the \element{schema} element.
+  \end{attributedesc}
+
+  \begin{attributedesc}{datatype}{\datatype{basic-key}
+                                  or \datatype{dotted-name}}
+    The data type converter which will be applied to the value of this
+    section.  If the value is a \datatype{dotted-name} that begins
+    with a period, the value of \attribute{prefix} will be pre-pended,
+    if set.  If any base schemas are listed in the \attribute{extends}
+    attribute, the default value for this attribute comes from the base
+    schemas.  If the base schemas all use the same \attribute{datatype}, then
+    that data type will be the default value for the extending schema.  If
+    there are no base schemas, the default value is \datatype{null}, which
+    means that the \module{ZConfig} section object will be used unconverted.
+    If the base schemas have different \attribute{datatype} definitions, you
+    must explicitly define the \attribute{datatype} in the extending schema.
+  \end{attributedesc}
+
+  \begin{attributedesc}{handler}{\datatype{basic-key}}
+  \end{attributedesc}
+
+  \begin{attributedesc}{keytype}{\datatype{basic-key}
+                                  or \datatype{dotted-name}}
+    The data type converter which will be applied to keys found in
+    this section.  This can be used to constrain key values in
+    different ways; two data types which may be especially useful are
+    the \datatype{identifier} and \datatype{ipaddr-or-hostname}
+    types.  If the value is a \datatype{dotted-name} that begins
+    with a period, the value of \attribute{prefix} will be pre-pended,
+    if set.  If any base schemas are listed in the \attribute{extends}
+    attribute, the default value for this attribute comes from the base
+    schemas.  If the base schemas all use the same \attribute{keytype}, then
+    that key type will be the default value for the extending schema.  If there
+    are no base schemas, the default value is \datatype{basic-key}.  If the
+    base schemas have different \attribute{keytype} definitions, you must
+    explicitly define the \attribute{keytype} in the extending schema.
+  \end{attributedesc}
+
+  \begin{attributedesc}{prefix}{\datatype{dotted-name}}
+    Prefix to be pre-pended in front of partial dotted-names that
+    start with a period.  The value of this attribute is used in all
+    contexts with the \element{schema} element if it hasn't been
+    overridden by an inner element with a \attribute{prefix}
+    attribute.
+  \end{attributedesc}
+\end{elementdesc}
+
+\begin{elementdesc}{description}{PCDATA}
+  Descriptive text explaining the purpose the container of the
+  \element{description} element.  Most other elements can contain
+  a \element{description} element as their first child.
+  At most one \element{description} element may appear in a given
+  context.
+
+  \begin{attributedesc}{format}{NMTOKEN}
+    Optional attribute that can be added to indicate what conventions
+    are used to mark up the contained text.  This is intended to serve
+    as a hint for documentation extraction tools.  Suggested values
+    are:
+
+    \begin{tableii}{l|l}{code}{Value}{Content Format}
+      \lineii{plain}{\mimetype{text/plain}; blank lines separate paragraphs}
+      \lineii{rest}{reStructuredText}
+      \lineii{stx}{Classic Structured Text}
+    \end{tableii}
+  \end{attributedesc}
+\end{elementdesc}
+
+\begin{elementdesc}{example}{PCDATA}
+  An example value.  This serves only as documentation.
+\end{elementdesc}
+
+\begin{elementdesc}{metadefault}{PCDATA}
+  A description of the default value, for human readers.  This may
+  include information about how a computed value is determined when
+  the schema does not specify a default value.
+\end{elementdesc}
+
+\begin{elementdesc}{abstracttype}{description?}
+  Define an abstract section type.
+
+  \begin{attributedesc}{name}{\datatype{basic-key}}
+    The name of the abstract section type; required.
+  \end{attributedesc}
+\end{elementdesc}
+
+\begin{elementdesc}{sectiontype}{description?, (section | key |
+                                  multisection | multikey)*}
+  Define a concrete section type.
+
+  \begin{attributedesc}{datatype}{\datatype{basic-key}
+                                  or \datatype{dotted-name}}
+    The data type converter which will be applied to the value of this
+    section.  If the value is a \datatype{dotted-name} that begins
+    with a period, the value of \attribute{prefix} will be pre-pended,
+    if set.  If \attribute{datatype} is omitted and
+    \attribute{extends} is used, the \attribute{datatype} from the
+    section type identified by the \attribute{extends} attribute is
+    used.
+  \end{attributedesc}
+
+  \begin{attributedesc}{extends}{\datatype{basic-key}}
+    The name of a concrete section type from which this section type
+    acquires all key and section declarations.  This type does
+    \emph{not} automatically implement any abstract section type
+    implemented by the named section type.  If omitted, this section
+    is defined with only the keys and sections contained within the
+    \element{sectiontype} element.  The new section type is called a
+    \emph{derived} section type, and the type named by this attribute
+    is called the \emph{base} type.  Values for the
+    \attribute{datatype} and \attribute{keytype} attributes are
+    acquired from the base type if not specified.
+  \end{attributedesc}
+
+  \begin{attributedesc}{implements}{\datatype{basic-key}}
+    The name of an abstract section type which this concrete section
+    type implements.  If omitted, this section type does not implement
+    any abstract type, and can only be used if it is specified
+    directly in a schema or other section type.
+  \end{attributedesc}
+
+  \begin{attributedesc}{keytype}{\datatype{basic-key}}
+    The data type converter which will be applied to keys found in
+    this section.  This can be used to constrain key values in
+    different ways; two data types which may be especially useful are
+    the \datatype{identifier} and \datatype{ipaddr-or-hostname}
+    types.  If the value is a \datatype{dotted-name} that begins
+    with a period, the value of \attribute{prefix} will be pre-pended,
+    if set.  The default value is \datatype{basic-key}.  If
+    \attribute{keytype} is omitted and \attribute{extends} is used,
+    the \attribute{keytype} from the section type identified by the
+    \attribute{extends} attribute is used.
+  \end{attributedesc}
+
+  \begin{attributedesc}{name}{\datatype{basic-key}}
+    The name of the section type; required.
+  \end{attributedesc}
+
+  \begin{attributedesc}{prefix}{\datatype{dotted-name}}
+    Prefix to be pre-pended in front of partial dotted-names that
+    start with a period.  The value of this attribute is used in all
+    contexts in the \element{sectiontype} element.  If omitted, the
+    prefix specified by a containing context is used if specified.
+  \end{attributedesc}
+\end{elementdesc}
+
+\begin{elementdesc}{import}{EMPTY}
+  Import a schema component.  Exactly one of the attributes
+  \attribute{package} and \attribute{src} must be specified.
+
+  \begin{attributedesc}{file}{file name without directory information}
+    Name of the component file within a package; if not specified,
+    \file{component.xml} is used.  This may only be given when
+    \attribute{package} is used.  (The \file{component.xml} file is
+    always used when importing via \keyword{\%import} from a
+    configuration file.)
+  \end{attributedesc}
+
+  \begin{attributedesc}{package}{\datatype{dotted-suffix}}
+    Name of a Python package that contains the schema component being
+    imported.  The component will be loaded from the file identified
+    by the \attribute{file} attribute, or \file{component.xml} if
+    \attribute{file} is not specified.  If the package name given
+    starts with a dot (\character{.}), the name used will be the
+    current prefix and the value of this attribute concatenated.
+  \end{attributedesc}
+
+  \begin{attributedesc}{src}{\datatype{url-reference}}
+    URL to a separate schema which can provide useful types.  The
+    referenced resource must contain a schema, not a schema
+    component.  Section types defined or imported by the referenced
+    schema are added to the schema containing the \element{import};
+    top-level keys and sections are ignored.
+  \end{attributedesc}
+\end{elementdesc}
+
+\begin{elementdesc}{key}{description?, example?, metadefault?, default*}
+  A \element{key} element is used to describe a key-value pair which
+  may occur at most once in the section type or top-level schema in
+  which it is listed.
+
+  \begin{attributedesc}{attribute}{\datatype{identifier}}
+    The name of the Python attribute which this key should be the
+    value of on a \class{SectionValue} instance.  This must be unique
+    within the immediate contents of a section type or schema.  If
+    this attribute is not specified, an attribute name will be
+    computed by converting hyphens in the key name to underscores.
+  \end{attributedesc}
+
+  \begin{attributedesc}{datatype}{\datatype{basic-key}
+                                  or \datatype{dotted-name}}
+    The data type converter which will be applied to the value of this
+    key.  If the value is a \datatype{dotted-name} that begins
+    with a period, the value of \attribute{prefix} will be pre-pended,
+    if set.
+  \end{attributedesc}
+
+  \begin{attributedesc}{default}{\datatype{string}}
+    If the key-value pair is optional and this attribute is specified,
+    the value of this attribute will be converted using the appropriate
+    data type converter and returned to the application as the
+    configured value.  This attribute may not be specified if the
+    \attribute{required} attribute is \code{yes}.
+  \end{attributedesc}
+
+  \begin{attributedesc}{handler}{\datatype{basic-key}}
+  \end{attributedesc}
+
+  \begin{attributedesc}{name}{\datatype{basic-key}}
+    The name of the key, as it must be given in a configuration
+    instance, or `\code{*}'.  If the value is `\code{*}', any name not
+    already specified as a key may be used, and the configuration
+    value for the key will be a dictionary mapping from the key name
+    to the value.  In this case, the \attribute{attribute} attribute
+    must be specified, and the data type for the key will be applied
+    to each key which is found.
+  \end{attributedesc}
+
+  \begin{attributedesc}{required}{\code{yes|no}}
+    Specifies whether the configuration instance is required to
+    provide the key.  If the value is \code{yes}, the
+    \attribute{default} attribute may not be specified and an error
+    will be reported if the configuration instance does not specify a
+    value for the key.  If the value is \code{no} (the default) and
+    the configuration instance does not specify a value, the value
+    reported to the application will be that specified by the
+    \attribute{default} attribute, if given, or \code{None}.
+  \end{attributedesc}
+\end{elementdesc}
+
+
+\begin{elementdesc}{multikey}{description?, example?, metadefault?, default*}
+  A \element{multikey} element is used to describe a key-value pair
+  which may occur any number of times in the section type or top-level
+  schema in which it is listed.
+
+  \begin{attributedesc}{attribute}{\datatype{identifier}}
+    The name of the Python attribute which this key should be the
+    value of on a \class{SectionValue} instance.  This must be unique
+    within the immediate contents of a section type or schema.  If
+    this attribute is not specified, an attribute name will be
+    computed by converting hyphens in the key name to underscores.
+  \end{attributedesc}
+
+  \begin{attributedesc}{datatype}{\datatype{basic-key}
+                                  or \datatype{dotted-name}}
+    The data type converter which will be applied to the value of this
+    key.  If the value is a \datatype{dotted-name} that begins
+    with a period, the value of \attribute{prefix} will be pre-pended,
+    if set.
+  \end{attributedesc}
+
+  \begin{attributedesc}{handler}{\datatype{basic-key}}
+  \end{attributedesc}
+
+  \begin{attributedesc}{name}{\datatype{basic-key}}
+    The name of the key, as it must be given in a configuration
+    instance, or `\code{+}'.  If the value is `\code{+}', any name not
+    already specified as a key may be used, and the configuration
+    value for the key will be a dictionary mapping from the key name
+    to the value.  In this case, the \attribute{attribute} attribute
+    must be specified, and the data type for the key will be applied
+    to each key which is found.
+  \end{attributedesc}
+
+  \begin{attributedesc}{required}{\code{yes|no}}
+    Specifies whether the configuration instance is required to
+    provide the key.  If the value is \code{yes}, no \element{default}
+    elements may be specified and an error will be reported if the
+    configuration instance does not specify at least one value for the
+    key.  If the value is \code{no} (the default) and the
+    configuration instance does not specify a value, the value
+    reported to the application will be a list containing one element
+    for each \element{default} element specified as a child of the
+    \element{multikey}.  Each value will be individually converted
+    according to the \attribute{datatype} attribute.
+  \end{attributedesc}
+\end{elementdesc}
+
+
+\begin{elementdesc}{default}{PCDATA}
+  Each \element{default} element specifies a single default value for
+  a \element{multikey}.  This element can be repeated to produce a
+  list of individual default values.  The text contained in the
+  element will be passed to the datatype conversion for the
+  \element{multikey}.
+
+  \begin{attributedesc}{key}{key type of the containing sectiontype}
+    Key to associate with the default value.  This is only used for
+    defaults of a \element{key} or \element{multikey} with a
+    \attribute{name} of \code{+}; in that case this attribute is
+    required.  It is an error to use the \attribute{key} attribute
+    with a \element{default} element for a \element{multikey} with a
+    name other than \code{+}.
+
+    \begin{notice}[warning]
+      The datatype of this attribute is that of the section type
+      \emph{containing} the actual keys, not necessarily that of the
+      section type which defines the key.  If a derived section
+      overrides the key type of the base section type, the actual
+      key type used is that of the derived section.
+
+      This can lead to confusing errors in schemas, though the
+      \refmodule{ZConfig} package checks for this when the schema is
+      loaded.  This situation is particularly likely when a derived
+      section type uses a key type which collapses multiple default
+      keys which were not collapsed by the base section type.
+
+      Consider this example schema:
+
+\begin{verbatim}
+<schema>
+  <sectiontype name="base" keytype="identifier">
+    <key name="+" attribute="mapping">
+      <default key="foo">some value</default>
+      <default key="FOO">some value</default>
+    </key>
+  </sectiontype>
+
+  <sectiontype name="derived" keytype="basic-key"
+               extends="base"/>
+
+  <section type="derived" name="*" attribute="section"/>
+</schema>
+\end{verbatim}
+
+      When this schema is loaded, a set of defaults for the
+      \datatype{derived} section type is computed.  Since
+      \datatype{basic-key} is case-insensitive (everything is
+      converted to lower case), \samp{foo} and \samp{Foo} are both
+      converted to \samp{foo}, which clashes since \element{key} only
+      allows one value for each key.
+    \end{notice}
+  \end{attributedesc}
+\end{elementdesc}
+
+
+\begin{elementdesc}{section}{description?}
+  A \element{section} element is used to describe a section which may
+  occur at most once in the section type or top-level schema in which
+  it is listed.
+
+  \begin{attributedesc}{attribute}{\datatype{identifier}}
+    The name of the Python attribute which this section should be the
+    value of on a \class{SectionValue} instance.  This must be unique
+    within the immediate contents of a section type or schema.  If
+    this attribute is not specified, an attribute name will be
+    computed by converting hyphens in the section name to underscores,
+    in which case the \attribute{name} attribute may not be \code{*}
+    or \code{+}.
+  \end{attributedesc}
+
+  \begin{attributedesc}{handler}{\datatype{basic-key}}
+  \end{attributedesc}
+
+  \begin{attributedesc}{name}{\datatype{basic-key}}
+    The name of the section, as it must be given in a configuration
+    instance, \code{*}, or \code{+}.  If the value is \code{*}, any
+    name not already specified as a key may be used.  If the value is
+    \code{*} or \code{+}, the \attribute{attribute} attribute must be
+    specified.  If the value is \code{*}, any name is allowed, or the
+    name may be omitted.  If the value is \code{+}, any name is
+    allowed, but some name must be provided.
+  \end{attributedesc}
+
+  \begin{attributedesc}{required}{\code{yes|no}}
+    Specifies whether the configuration instance is required to
+    provide the section.  If the value is \code{yes}, an error will be
+    reported if the configuration instance does not include the
+    section.  If the value is \code{no} (the default) and the
+    configuration instance does not include the section, the value
+    reported to the application will be \code{None}.
+  \end{attributedesc}
+
+  \begin{attributedesc}{type}{\datatype{basic-key}}
+    The section type which matching sections must implement.  If the
+    value names an abstract section type, matching sections in the
+    configuration file must be of a type which specifies that it
+    implements the named abstract type.  If the name identifies a
+    concrete type, the section type must match exactly.
+  \end{attributedesc}
+\end{elementdesc}
+
+
+\begin{elementdesc}{multisection}{description?}
+  A \element{multisection} element is used to describe a section which
+  may occur any number of times in the section type or top-level
+  schema in which it is listed.
+
+  \begin{attributedesc}{attribute}{\datatype{identifier}}
+    The name of the Python attribute which matching sections should be
+    the value of on a \class{SectionValue} instance.  This is required
+    and must be unique within the immediate contents of a section type
+    or schema.  The \class{SectionValue} instance will contain a list
+    of matching sections.
+  \end{attributedesc}
+
+  \begin{attributedesc}{handler}{\datatype{basic-key}}
+  \end{attributedesc}
+
+  \begin{attributedesc}{name}{\datatype{basic-key}}
+    For a \element{multisection}, any name not already specified as a
+    key may be used.  If the value is \code{*} or \code{+}, the
+    \attribute{attribute} attribute must be specified.  If the value
+    is \code{*}, any name is allowed, or the name may be omitted.  If
+    the value is \code{+}, any name is allowed, but some name must be
+    provided.  No other value for the \attribute{name} attribute is
+    allowed for a \element{multisection}.
+  \end{attributedesc}
+
+  \begin{attributedesc}{required}{\code{yes|no}}
+    Specifies whether the configuration instance is required to
+    provide at least one matching section.  If the value is
+    \code{yes}, an error will be reported if the configuration
+    instance does not include the section.  If the value is \code{no}
+    (the default) and the configuration instance does not include the
+    section, the value reported to the application will be
+    \code{None}.
+  \end{attributedesc}
+
+  \begin{attributedesc}{type}{\datatype{basic-key}}
+    The section type which matching sections must implement.  If the
+    value names an abstract section type, matching sections in the
+    configuration file must be of types which specify that they
+    implement the named abstract type.  If the name identifies a
+    concrete type, the section type must match exactly.
+  \end{attributedesc}
+\end{elementdesc}
+
+
+\subsection{Schema Components \label{schema-components}}
+
+XXX need more explanation
+
+\module{ZConfig} supports schema components that can be
+provided by disparate components, and allows them to be knit together
+into concrete schema for applications.  Components cannot add
+additional keys or sections in the application schema.
+
+A schema \dfn{component} is allowed to define new abstract and
+section types.
+Components are identified using a dotted-name, similar to a Python
+module name.  For example, one component may be \code{zodb.storage}.
+
+Schema components are stored alongside application code since they
+directly reference datatype code.  Schema components are provided by
+Python packages.  The component definition is normally stored in the
+file \file{component.xml}; an alternate filename may be specified
+using the \attribute{file} attribute of the \element{import} element.
+Components imported using the \keyword{\%import} keyword from a
+configuration file must be named \file{component.xml}.
+The component defines the types provided by that component; it must
+have a \element{component} element as the document element.
+
+The following element is used as the document element for schema
+components.  Note that schema components do not allow keys and
+sections to be added to the top-level of a schema; they serve only to
+provide type definitions.
+
+\begin{elementdesc}{component}{description?, (abstracttype | sectiontype)*}
+  The top-level element for schema components.
+
+  \begin{attributedesc}{prefix}{\datatype{dotted-name}}
+    Prefix to be pre-pended in front of partial dotted-names that
+    start with a period.  The value of this attribute is used in all
+    contexts within the \element{component} element if it hasn't been
+    overridden by an inner element with a \attribute{prefix}
+    attribute.
+  \end{attributedesc}
+\end{elementdesc}
+
+
+\section{Standard \module{ZConfig} Datatypes\label{standard-datatypes}}
+
+There are a number of data types which can be identified using the
+\attribute{datatype} attribute on \element{key},
+\element{sectiontype}, and \element{schema} elements.
+Applications may extend the set of datatypes by calling the
+\method{register()} method of the data type registry being used or by
+using Python dotted-names to refer to conversion routines defined in
+code.
+
+The following data types are provided by the default type registry.
+
+\begin{definitions}
+\term{\datatype{basic-key}}
+  The default data type for a key in a ZConfig configuration file.
+  The result of conversion is always lower-case, and matches the
+  regular expression \regexp{[a-z][-._a-z0-9]*}.
+
+\term{\datatype{boolean}}
+  Convert a human-friendly string to a boolean value.  The names
+  \code{yes}, \code{on}, and \code{true} convert to \constant{True},
+  while \code{no}, \code{off}, and \code{false} convert to
+  \constant{False}.  Comparisons are case-insensitive.  All other
+  input strings are disallowed.
+
+\term{\datatype{byte-size}}
+  A specification of a size, with byte multiplier suffixes (for
+  example, \samp{128MB}).  Suffixes are case insensitive and may be
+  \samp{KB}, \samp{MB}, or \samp{GB}
+
+\term{\datatype{dotted-name}}
+  A string consisting of one or more \datatype{identifier} values
+  separated by periods (\character{.}).
+
+\term{\datatype{dotted-suffix}}
+  A string consisting of one or more \datatype{identifier} values
+  separated by periods (\character{.}), possibly prefixed by a
+  period.  This can be used to indicate a dotted name that may be
+  specified relative to some base dotted name.
+
+\term{\datatype{existing-dirpath}}
+  Validates that the directory portion of a pathname exists.  For
+  example, if the value provided is \file{/foo/bar}, \file{/foo} must
+  be an existing directory.  No conversion is performed.
+
+\term{\datatype{existing-directory}}
+  Validates that a directory by the given name exists on 
+  the local filesystem.  No conversion is performed. 
+
+\term{\datatype{existing-file}}
+  Validates that a file by the given name exists.  No conversion 
+  is performed. 
+
+\term{\datatype{existing-path}}
+  Validates that a path (file, directory, or symlink) by the
+  given name exists on the local filesystem.  No conversion
+  is performed.
+
+\term{\datatype{float}}
+  A Python float.  \code{Inf}, \code{-Inf}, and \code{NaN} are not
+  allowed.
+
+\term{\datatype{identifier}}
+  Any valid Python identifier.
+
+\term{\datatype{inet-address}}
+  An Internet address expressed as a \code{(\var{hostname},
+  \var{port})} pair.  If only the port is specified, the default host
+  will be returned for \var{hostname}.  The default host is
+  \code{localhost} on Windows and the empty string on all other
+  platforms.  If the port is omitted, \code{None} will be returned for
+  \var{port}.
+
+\term{\datatype{integer}}
+  Convert a value to an integer.  This will be a Python \class{int} if
+  the value is in the range allowed by \class{int}, otherwise a Python
+  \class{long} is returned.
+
+\term{\datatype{ipaddr-or-hostname}}
+  Validates a valid IP address or hostname.  If the first 
+  character is a digit, the value is assumed to be an IP 
+  address.  If the first character is not a digit, the value 
+  is assumed to be a hostname.  Hostnames are converted to lower
+  case.
+
+\term{\datatype{locale}}
+  Any valid locale specifier accepted by the available
+  \function{locale.setlocale()} function.  Be aware that only the
+  \code{'C'} locale is supported on some platforms.
+
+\term{\datatype{null}}
+  No conversion is performed; the value passed in is the value
+  returned.  This is the default data type for section values.
+
+\term{\datatype{port-number}}
+  Returns a valid port number as an integer.  Validity does not imply
+  that any particular use may be made of the port, however.  For
+  example, port number lower than 1024 generally cannot be bound by
+  non-root users.
+
+\term{\datatype{socket-address}}
+  An address for a socket.  The converted value is an object providing
+  two attributes.  \member{family} specifies the address family
+  (\constant{AF_INET} or \constant{AF_UNIX}), with \code{None} instead
+  of \constant{AF_UNIX} on platforms that don't support it.  The
+  \member{address} attribute will be the address that should be passed
+  to the socket's \method{bind()} method.  If the family is
+  \constant{AF_UNIX}, the specific address will be a pathname; if the
+  family is \constant{AF_INET}, the second part will be the result of
+  the \datatype{inet-address} conversion.
+
+\term{\datatype{string}}
+  Returns the input value as a string.  If the source is a Unicode
+  string, this implies that it will be checked to be simple 7-bit
+  \ASCII.  This is the default data type for values in
+  configuration files.
+
+\term{\datatype{time-interval}}
+  A specification of a time interval in seconds, with multiplier
+  suffixes (for example, \code{12h}).  Suffixes are case insensitive
+  and may be \samp{s} (seconds), \samp{m} (minutes), \samp{h} (hours),
+  or \samp{d} (days).
+
+\term{\datatype{timedelta}}
+  Similar to the \datatype{time-interval}, this data type returns a Python
+  datetime.timedelta object instead of a float.  The set of suffixes
+  recognized by \datatype{timedelta} are: \samp{w} (weeks), \samp{d} (days),
+  \samp{h} (hours), \samp{m} (minutes), \samp{s} (seconds).  Values may be
+  floats, for example: \code{4w 2.5d 7h 12m 0.001s}.
+
+\end{definitions}
+
+
+\section{Standard \module{ZConfig} Schema Components
+         \label{standard-components}}
+
+\module{ZConfig} provides a few convenient schema components as part
+of the package.  These may be used directly or can server as examples
+for creating new components.
+
+
+\subsection{\module{ZConfig.components.basic}}
+
+The \module{ZConfig.components.basic} package provides small
+components that can be helpful in composing application-specific
+components and schema.  There is no large functionality represented by
+this package.  The default component provided by this package simply
+imports all of the smaller components.  This can be imported using
+
+\begin{verbatim}
+<import package="ZConfig.components.basic"/>
+\end{verbatim}
+
+Each of the smaller components is documented directly; importing these
+selectively can reduce the time it takes to load a schema slightly,
+and allows replacing the other basic components with alternate
+components (by using different imports that define the same type
+names) if desired.
+
+
+\subsubsection{The Mapping Section Type \label{basic-mapping}}
+
+There is a basic section type that behaves like a simple Python
+mapping; this can be imported directly using
+
+\begin{verbatim}
+<import package="ZConfig.components.basic" file="mapping.xml"/>
+\end{verbatim}
+
+This defines a single section type, \datatype{ZConfig.basic.mapping}.
+When this is used, the section value is a Python dictionary mapping
+keys to string values.
+
+This type is intended to be used by extending it in simple ways.  The
+simplest is to create a new section type name that makes more sense
+for the application:
+
+\begin{verbatim}
+<import package="ZConfig.components.basic" file="mapping.xml"/>
+
+<sectiontype name="my-mapping"
+             extends="ZConfig.basic.mapping"
+             />
+
+<section name="*"
+         type="my-mapping"
+         attribute="map"
+         />
+\end{verbatim}
+
+This allows a configuration to contain a mapping from
+\datatype{basic-key} names to string values like this:
+
+\begin{verbatim}
+<my-mapping>
+  This that
+  and the other
+</my-mapping>
+\end{verbatim}
+
+The value of the configuration object's \member{map} attribute would
+then be the dictionary
+
+\begin{verbatim}
+{'this': 'that',
+ 'and': 'the other',
+ }
+\end{verbatim}
+
+(Recall that the \datatype{basic-key} data type converts everything to
+lower case.)
+
+Perhaps a more interesting application of
+\datatype{ZConfig.basic.mapping} is using the derived type to override
+the \attribute{keytype}.  If we have the conversion function:
+
+\begin{verbatim}
+def email_address(value):
+    userid, hostname = value.split("@", 1)
+    hostname = hostname.lower()  # normalize what we know we can
+    return "%s@%s" % (userid, hostname)
+\end{verbatim}
+
+then we can use this as the key type for a derived mapping type:
+
+\begin{verbatim}
+<import package="ZConfig.components.basic" file="mapping.xml"/>
+
+<sectiontype name="email-users"
+             extends="ZConfig.basic.mapping"
+             keytype="mypkg.datatypes.email_address"
+             />
+
+<section name="*"
+         type="email-users"
+         attribute="email_users"
+         />
+\end{verbatim}
+
+
+\subsection{\module{ZConfig.components.logger}}
+
+The \module{ZConfig.components.logger} package provides configuration
+support for the \ulink{\module{logging} package}
+{http://docs.python.org/lib/module-logging.html} in
+Python's standard library.  This component can be imported using
+
+\begin{verbatim}
+<import package="ZConfig.components.logger"/>
+\end{verbatim}
+
+This component defines two abstract types and several concrete section
+types.  These can be imported as a unit, as above, or as four smaller
+components usable in creating alternate logging packages.
+
+The first of the four smaller components contains the abstract types,
+and can be imported using
+
+\begin{verbatim}
+<import package="ZConfig.components.logger" file="abstract.xml"/>
+\end{verbatim}
+
+The two abstract types imported by this are:
+
+\begin{definitions}
+
+\term{\datatype{ZConfig.logger.log}}
+  Logger objects are represented by this abstract type.
+
+\term{\datatype{ZConfig.logger.handler}}
+  Each logger object can have one or more ``handlers'' associated with
+  them.  These handlers are responsible for writing logging events to
+  some form of output stream using appropriate formatting.  The output
+  stream may be a file on a disk, a socket communicating with a server
+  on another system, or a series of \code{syslog} messages.  Section
+  types which implement this type represent these handlers.
+
+\end{definitions}
+
+
+The second and third of the smaller components provides section types
+that act as factories for \class{logging.Logger} objects.  These can be
+imported using
+
+\begin{verbatim}
+<import package="ZConfig.components.logger" file="eventlog.xml"/>
+<import package="ZConfig.components.logger" file="logger.xml"/>
+\end{verbatim}
+
+The types defined in these components implement the
+\datatype{ZConfig.logger.log} abstract type.  The \file{eventlog.xml}
+component defines an \datatype{eventlog} type which represents the
+root logger from the the \module{logging} package (the return value of
+\function{logging.getLogger()}), while the \file{logger.xml} component
+defines a \datatype{logger} section type which represents a named
+logger (as returned by \function{logging.getLogger(\var{name})}).
+
+
+The third of the smaller components provides section types that are
+factories for \class{logging.Handler} objects.  This can be imported
+using
+
+\begin{verbatim}
+<import package="ZConfig.components.logger" file="handlers.xml"/>
+\end{verbatim}
+
+The types defined in this component implement the
+\datatype{ZConfig.logger.handler} abstract type.
+
+
+
+The configuration objects provided by both the logger and handler
+types are factories for the finished loggers and handlers.  These
+factories should be called with no arguments to retrieve the logger or
+log handler objects.  Calling the factories repeatedly will cause the
+same objects to be returned each time, so it's safe to simply call
+them to retrieve the objects.
+
+The factories for the logger objects, whether the \datatype{eventlog}
+or \datatype{logger} section type is used, provide a \method{reopen()}
+method which may be called to close any log files and re-open them.
+This is useful when using a \UNIX{} signal to effect log file
+rotation: the signal handler can call this method, and not have to
+worry about what handlers have been registered for the logger.
+
+Building an application that uses the logging components is fairly
+straightforward.  The schema needs to import the relevant components
+and declare their use:
+
+\begin{verbatim}
+<schema>
+  <import package="ZConfig.components.logger" file="eventlog.xml"/>
+  <import package="ZConfig.components.logger" file="handlers.xml"/>
+
+  <section type="eventlog" name="*" attribute="eventlog"
+           required="yes"/>
+</schema>
+\end{verbatim}
+
+In the application, the schema and configuration file should be loaded
+normally.  Once the configuration object is available, the logger
+factory should be called to configure Python's \module{logging} package:
+
+\begin{verbatim}
+import os
+import ZConfig
+
+def run(configfile):
+    schemafile = os.path.join(os.path.dirname(__file__), "schema.xml")
+    schema = ZConfig.loadSchema(schemafile)
+    config, handlers = ZConfig.loadConfig(schema, configfile)
+
+    # configure the logging package:
+    config.eventlog()
+
+    # now do interesting things
+\end{verbatim}
+
+An example configuration file for this application may look like this:
+
+\begin{verbatim}
+<eventlog>
+  level  info
+
+  <logfile>
+    path        /var/log/myapp
+    format      %(asctime)s %(levelname)s %(name)s %(message)s
+    # locale-specific date/time representation
+    dateformat  %c
+  </logfile>
+
+  <syslog>
+    level    error
+    address  syslog.example.net:514
+    format   %(levelname)s %(name)s %(message)s
+  </syslog>
+</eventlog>
+\end{verbatim}
+
+Refer to the \module{logging} package documentation for the names
+available in the message format strings (the \code{format} key in the
+log handlers).  The date format strings (the \code{dateformat} key in
+the log handlers) are the same as those accepted by the
+\function{time.strftime()} function.
+
+
+\begin{seealso}
+  \seepep{282}{A Logging System}
+         {The proposal which described the logging feature for
+          inclusion in the Python standard library.}
+  \seelink{http://docs.python.org/lib/module-logging.html}
+          {\module{logging} --- Logging facility for Python}
+          {Python's \module{logging} package documentation, from the
+           \citetitle[http://docs.python.org/lib/lib.html]
+           {Python Library Reference}.}
+  \seelink{http://www.red-dove.com/python_logging.html}
+          {Original Python \module{logging} package}
+          {This is the original source for the \module{logging}
+           package.  This is mostly of historical interest.}
+\end{seealso}
+
+
+\section{Using Components to Extend Schema}
+
+% XXX This section needs a lot of work, but should get people started
+% who really want to add new pieces to ZConfig-configured applications.
+
+It is possible to use schema components and the \keyword{\%import}
+construct to extend the set of section types available for a specific
+configuration file, and allow the new components to be used in place
+of standard components.
+
+The key to making this work is the use of abstract section types.
+Wherever the original schema accepts an abstract type, it is possible
+to load new implementations of the abstract type and use those instead
+of, or in addition to, the implementations loaded by the original
+schema.
+
+Abstract types are generally used to represent interfaces.  Sometimes
+these are interfaces for factory objects, and sometimes not, but
+there's an interface that the new component needs to implement.  What
+interface is required should be documented in the
+\element{description} element in the \element{abstracttype} element;
+this may be by reference to an interface specified in a Python module
+or described in some other bit of documentation.
+
+The following things need to be created to make the new component
+usable from the configuration file:
+
+\begin{enumerate}
+  \item An implementation of the required interface.
+
+  \item A schema component that defines a section type that contains
+        the information needed to construct the component.
+
+  \item A ``datatype'' function that converts configuration data to an
+        instance of the component.
+\end{enumerate}
+
+For simplicity, let's assume that the implementation is defined by a
+Python class.
+
+The example component we build here will be in the \module{noise}
+package, but any package will do.  Components loadable using
+\keyword{\%import} must be contained in the \file{component.xml} file;
+alternate filenames may not be selected by the \keyword{\%import}
+construct.
+
+Create a ZConfig component that provides a section type to support
+your component.  The new section type must declare that it implements
+the appropriate abstract type; it should probably look something like
+this:
+
+\begin{verbatim}
+<component prefix="noise.server">
+  <import package="ZServer"/>
+
+  <sectiontype name="noise-generator"
+               implements="ZServer.server"
+               datatype=".NoiseServerFactory">
+
+    <!-- specific configuration data should be described here -->
+
+    <key name="port"
+         datatype="port-number"
+         required="yes">
+      <description>
+        Port number to listen on.
+      </description>
+    </key>
+
+    <key name="color"
+         datatype=".noise_color"
+         default="white">
+      <description>
+        Silly way to specify a noise generation algorithm.
+      </description>
+    </key>
+
+  </sectiontype>
+</component>
+\end{verbatim}
+
+This example uses one of the standard ZConfig datatypes,
+\datatype{port-number}, and requires two additional types to be
+provided by the \module{noise.server} module:
+\class{NoiseServerFactory} and \function{noise_color()}.
+
+The \function{noise_color()} function is a datatype conversion for a
+key, so it accepts a string and returns the value that should be used:
+
+\begin{verbatim}
+_noise_colors = {
+    # color -> r,g,b
+    'white': (255, 255, 255),
+    'pink':  (255, 182, 193),
+    }
+
+def noise_color(string):
+    if string in _noise_colors:
+        return _noise_colors[string]
+    else:
+        raise ValueError('unknown noise color: %r' % string)
+\end{verbatim}
+
+\class{NoiseServerFactory} is a little different, as it's the datatype
+function for a section rather than a key.  The parameter isn't a
+string, but a section value object with two attributes, \member{port}
+and \member{color}.
+
+Since the \datatype{ZServer.server} abstract type requires that the
+component returned is a factory object, the datatype function can be
+implemented at the constructor for the class of the factory object.
+(If the datatype function could select different implementation
+classes based on the configuration values, it makes more sense to use
+a simple function that returns the appropriate implementation.)
+
+A class that implements this datatype might look like this:
+
+\begin{verbatim}
+from ZServer.datatypes import ServerFactory
+from noise.generator import WhiteNoiseGenerator, PinkNoiseGenerator
+
+class NoiseServerFactory(ServerFactory):
+
+    def __init__(self, section):
+        # host and ip will be initialized by ServerFactory.prepare()
+        self.host = None
+        self.ip = None
+        self.port = section.port
+        self.color = section.color
+
+    def create(self):
+        if self.color == 'white':
+            generator = WhiteNoiseGenerator()
+        else:
+            generator = PinkNoiseGenerator()
+        return NoiseServer(self.ip, self.port, generator)
+\end{verbatim}
+
+You'll need to arrange for the package containing this component is
+available on Python's \code{sys.path} before the configuration file is
+loaded; this is mostly easily done by manipulating the
+\envvar{PYTHONPATH} environment variable.
+
+Your configuration file can now include the following to load and use
+your new component:
+
+\begin{verbatim}
+%import noise
+
+<noise-generator>
+  port 1234
+  color white
+</noise-generator>
+\end{verbatim}
+
+
+\section{\module{ZConfig} --- Basic configuration support}
+
+\declaremodule{}{ZConfig}
+\modulesynopsis{Configuration package.}
+
+The main \module{ZConfig} package exports these convenience functions:
+
+\begin{funcdesc}{loadConfig}{schema, url\optional{, overrides}}
+  Load and return a configuration from a URL or pathname given by
+  \var{url}.  \var{url} may be a URL, absolute pathname, or relative
+  pathname.  Fragment identifiers are not supported.  \var{schema} is
+  a reference to a schema loaded by \function{loadSchema()} or
+  \function{loadSchemaFile()}.
+  The return value is a tuple containing the configuration object and
+  a composite handler that, when called with a name-to-handler
+  mapping, calls all the handlers for the configuration.
+
+  The optional \var{overrides} argument represents information derived
+  from command-line arguments.  If given, it must be either a sequence
+  of value specifiers, or \code{None}.  A \dfn{value specifier} is a
+  string of the form \code{\var{optionpath}=\var{value}}.  The
+  \var{optionpath} specifies the ``full path'' to the configuration
+  setting: it can contain a sequence of names, separated by
+  \character{/} characters. Each name before the last names a section
+  from the configuration file, and the last name corresponds to a key
+  within the section identified by the leading section names.  If
+  \var{optionpath} contains only one name, it identifies a key in the
+  top-level schema.  \var{value} is a string that will be treated
+  just like a value in the configuration file.
+\end{funcdesc}
+
+\begin{funcdesc}{loadConfigFile}{schema, file\optional{,
+                                 url\optional{, overrides}}}
+  Load and return a configuration from an opened file object.  If
+  \var{url} is omitted, one will be computed based on the
+  \member{name} attribute of \var{file}, if it exists.  If no URL can
+  be determined, all \keyword{\%include} statements in the
+  configuration must use absolute URLs.  \var{schema} is a reference
+  to a schema loaded by \function{loadSchema()} or
+  \function{loadSchemaFile()}.
+  The return value is a tuple containing the configuration object and
+  a composite handler that, when called with a name-to-handler
+  mapping, calls all the handlers for the configuration.
+  The \var{overrides} argument is the same as for the
+  \function{loadConfig()} function.
+\end{funcdesc}
+
+\begin{funcdesc}{loadSchema}{url}
+  Load a schema definition from the URL \var{url}.
+  \var{url} may be a URL, absolute pathname, or relative pathname.
+  Fragment identifiers are not supported.
+  The resulting
+  schema object can be passed to \function{loadConfig()} or
+  \function{loadConfigFile()}.  The schema object may be used as many
+  times as needed.
+\end{funcdesc}
+
+\begin{funcdesc}{loadSchemaFile}{file\optional{, url}}
+  Load a schema definition from the open file object \var{file}.  If
+  \var{url} is given and not \code{None}, it should be the URL of
+  resource represented by \var{file}.  If \var{url} is omitted or
+  \code{None}, a URL may be computed from the \member{name} attribute
+  of \var{file}, if present.  The resulting schema object can
+  be passed to \function{loadConfig()} or \function{loadConfigFile()}.
+  The schema object may be used as many times as needed.
+\end{funcdesc}
+
+The following exceptions are defined by this package:
+
+\begin{excdesc}{ConfigurationError}
+  Base class for exceptions specific to the \module{ZConfig} package.
+  All instances provide a \member{message} attribute that describes
+  the specific error, and a \member{url} attribute that gives the URL
+  of the resource the error was located in, or \constant{None}.
+\end{excdesc}
+
+\begin{excdesc}{ConfigurationSyntaxError}
+  Exception raised when a configuration source does not conform to the
+  allowed syntax.  In addition to the \member{message} and
+  \member{url} attributes, exceptions of this type offer the
+  \member{lineno} attribute, which provides the line number at which
+  the error was detected.
+\end{excdesc}
+
+\begin{excdesc}{DataConversionError}
+  Raised when a data type conversion fails with
+  \exception{ValueError}.  This exception is a subclass of both
+  \exception{ConfigurationError} and \exception{ValueError}.  The
+  \function{str()} of the exception provides the explanation from the
+  original \exception{ValueError}, and the line number and URL of the
+  value which provoked the error.  The following additional attributes
+  are provided:
+
+  \begin{tableii}{l|l}{member}{Attribute}{Value}
+    \lineii{colno}
+           {column number at which the value starts, or \code{None}}
+    \lineii{exception}
+           {the original \exception{ValueError} instance}
+    \lineii{lineno}
+           {line number on which the value starts}
+    \lineii{message}
+           {\function{str()} returned by the original \exception{ValueError}}
+    \lineii{value}
+           {original value passed to the conversion function}
+    \lineii{url}
+           {URL of the resource providing the value text}
+  \end{tableii}
+\end{excdesc}
+
+\begin{excdesc}{SchemaError}
+  Raised when a schema contains an error.  This exception type
+  provides the attributes \member{url}, \member{lineno}, and
+  \member{colno}, which provide the source URL, the line number, and
+  the column number at which the error was detected.  These attributes
+  may be \code{None} in some cases.
+\end{excdesc}
+
+\begin{excdesc}{SchemaResourceError}
+  Raised when there's an error locating a resource required by the
+  schema.  This is derived from \exception{SchemaError}.  Instances of
+  this exception class add the attributes \member{filename},
+  \member{package}, and \member{path}, which hold the filename
+  searched for within the package being loaded, the name of the
+  package, and the \code{__path__} attribute of the package itself (or
+  \constant{None} if it isn't a package or could not be imported).
+\end{excdesc}
+
+\begin{excdesc}{SubstitutionReplacementError}
+  Raised when the source text contains references to names which are
+  not defined in \var{mapping}.  The attributes \member{source} and
+  \member{name} provide the complete source text and the name
+  (converted to lower case) for which no replacement is defined.
+\end{excdesc}
+
+\begin{excdesc}{SubstitutionSyntaxError}
+  Raised when the source text contains syntactical errors.
+\end{excdesc}
+
+
+\subsection{Basic Usage}
+
+The simplest use of \refmodule{ZConfig} is to load a configuration
+based on a schema stored in a file.  This example loads a
+configuration file specified on the command line using a schema in the
+same directory as the script:
+
+\begin{verbatim}
+import os
+import sys
+import ZConfig
+
+try:
+    myfile = __file__
+except NameError:
+    myfile = os.path.realpath(sys.argv[0])
+
+mydir = os.path.dirname(myfile)
+
+schema = ZConfig.loadSchema(os.path.join(mydir, 'schema.xml'))
+conf, handler = ZConfig.loadConfig(schema, sys.argv[1])
+\end{verbatim}
+
+If the schema file contained this schema:
+
+\begin{verbatim}
+<schema>
+  <key name='server' required='yes'/>
+  <key name='attempts' datatype='integer' default='5'/>
+</schema>
+\end{verbatim}
+
+and the file specified on the command line contained this text:
+
+\begin{verbatim}
+# sample configuration
+
+server www.example.com
+\end{verbatim}
+
+then the configuration object \code{conf} loaded above would have two
+attributes:
+
+\begin{tableii}{l|l}{member}{Attribute}{Value}
+  \lineii{server}{\code{'www.example.com'}}
+  \lineii{attempts}{\code{5}}
+\end{tableii}
+
+
+\section{\module{ZConfig.datatypes} --- Default data type registry}
+
+\declaremodule{}{ZConfig.datatypes}
+\modulesynopsis{Default implementation of a data type registry}
+
+The \module{ZConfig.datatypes} module provides the implementation of
+the default data type registry and all the standard data types
+supported by \module{ZConfig}.  A number of convenience classes are
+also provided to assist in the creation of additional data types.
+
+A \dfn{datatype registry} is an object that provides conversion
+functions for data types.  The interface for a registry is fairly
+simple.
+
+A \dfn{conversion function} is any callable object that accepts a
+single argument and returns a suitable value, or raises an exception
+if the input value is not acceptable.  \exception{ValueError} is the
+preferred exception for disallowed inputs, but any other exception
+will be properly propagated.
+
+\begin{classdesc}{Registry}{\optional{stock}}
+  Implementation of a simple type registry.  If given, \var{stock}
+  should be a mapping which defines the ``built-in'' data types for
+  the registry; if omitted or \code{None}, the standard set of data
+  types is used (see section~\ref{standard-datatypes}, ``Standard
+  \module{ZConfig} Datatypes'').
+\end{classdesc}
+
+\class{Registry} objects have the following methods:
+
+\begin{methoddesc}{get}{name}
+  Return the type conversion routine for \var{name}.  If the
+  conversion function cannot be found, an (unspecified) exception is
+  raised.  If the name is not provided in the stock set of data types
+  by this registry and has not otherwise been registered, this method
+  uses the \method{search()} method to load the conversion function.
+  This is the only method the rest of \module{ZConfig} requires.
+\end{methoddesc}
+
+\begin{methoddesc}{register}{name, conversion}
+  Register the data type name \var{name} to use the conversion
+  function \var{conversion}.  If \var{name} is already registered or
+  provided as a stock data type, \exception{ValueError} is raised
+  (this includes the case when \var{name} was found using the
+  \method{search()} method).
+\end{methoddesc}
+
+\begin{methoddesc}{search}{name}
+  This is a helper method for the default implementation of the
+  \method{get()} method.  If \var{name} is a Python dotted-name, this
+  method loads the value for the name by dynamically importing the
+  containing module and extracting the value of the name.  The name
+  must refer to a usable conversion function.
+\end{methoddesc}
+
+
+The following classes are provided to define conversion functions:
+
+\begin{classdesc}{MemoizedConversion}{conversion}
+  Simple memoization for potentially expensive conversions.  This
+  conversion helper caches each successful conversion for re-use at a
+  later time; failed conversions are not cached in any way, since it
+  is difficult to raise a meaningful exception providing information
+  about the specific failure.
+\end{classdesc}
+
+\begin{classdesc}{RangeCheckedConversion}{conversion\optional{,
+                                          min\optional{, max}}}
+  Helper that performs range checks on the result of another
+  conversion.  Values passed to instances of this conversion are
+  converted using \var{conversion} and then range checked.  \var{min}
+  and \var{max}, if given and not \code{None}, are the inclusive
+  endpoints of the allowed range.  Values returned by \var{conversion}
+  which lay outside the range described by \var{min} and \var{max}
+  cause \exception{ValueError} to be raised.
+\end{classdesc}
+
+\begin{classdesc}{RegularExpressionConversion}{regex}
+  Conversion that checks that the input matches the regular expression
+  \var{regex}.  If it matches, returns the input, otherwise raises
+  \exception{ValueError}.
+\end{classdesc}
+
+
+\section{\module{ZConfig.loader} --- Resource loading support}
+
+\declaremodule{}{ZConfig.loader}
+\modulesynopsis{Support classes for resource loading}
+
+This module provides some helper classes used by the primary APIs
+exported by the \module{ZConfig} package.  These classes may be useful
+for some applications, especially applications that want to use a
+non-default data type registry.
+
+\begin{classdesc}{Resource}{file, url\optional{, fragment}}
+  Object that allows an open file object and a URL to be bound
+  together to ease handling.  Instances have the attributes
+  \member{file}, \member{url}, and \member{fragment} which store the
+  constructor arguments.  These objects also have a \method{close()}
+  method which will call \method{close()} on \var{file}, then set the
+  \member{file} attribute to \code{None} and the \member{closed} to
+  \constant{True}.
+\end{classdesc}
+
+\begin{classdesc}{BaseLoader}{}
+  Base class for loader objects.  This should not be instantiated
+  directly, as the \method{loadResource()} method must be overridden
+  for the instance to be used via the public API.
+\end{classdesc}
+
+\begin{classdesc}{ConfigLoader}{schema}
+  Loader for configuration files.  Each configuration file must
+  conform to the schema \var{schema}.  The \method{load*()} methods
+  return a tuple consisting of the configuration object and a
+  composite handler.
+\end{classdesc}
+
+\begin{classdesc}{SchemaLoader}{\optional{registry}}
+  Loader that loads schema instances.  All schema loaded by a
+  \class{SchemaLoader} will use the same data type registry.  If
+  \var{registry} is provided and not \code{None}, it will be used,
+  otherwise an instance of \class{ZConfig.datatypes.Registry} will be
+  used.
+\end{classdesc}
+
+
+\subsection{Loader Objects}
+
+Loader objects provide a general public interface, an interface which
+subclasses must implement, and some utility methods.
+
+The following methods provide the public interface:
+
+\begin{methoddesc}[loader]{loadURL}{url}
+  Open and load a resource specified by the URL \var{url}.
+  This method uses the \method{loadResource()} method to perform the
+  actual load, and returns whatever that method returns.
+\end{methoddesc}
+
+\begin{methoddesc}[loader]{loadFile}{file\optional{, url}}
+  Load from an open file object, \var{file}.  If given and not
+  \code{None}, \var{url} should be the URL of the resource represented
+  by \var{file}.  If omitted or \code{None}, the \member{name}
+  attribute of \var{file} is used to compute a \code{file:} URL, if
+  present.
+  This method uses the \method{loadResource()} method to perform the
+  actual load, and returns whatever that method returns.
+\end{methoddesc}
+
+The following method must be overridden by subclasses:
+
+\begin{methoddesc}[loader]{loadResource}{resource}
+  Subclasses of \class{BaseLoader} must implement this method to
+  actually load the resource and return the appropriate
+  application-level object.
+\end{methoddesc}
+
+The following methods can be used as utilities:
+
+\begin{methoddesc}[loader]{isPath}{s}
+  Return true if \var{s} should be considered a filesystem path rather
+  than a URL.
+\end{methoddesc}
+
+\begin{methoddesc}[loader]{normalizeURL}{url-or-path}
+  Return a URL for \var{url-or-path}.  If \var{url-or-path} refers to
+  an existing file, the corresponding \code{file:} URL is returned.
+  Otherwise \var{url-or-path} is checked for sanity: if it
+  does not have a schema, \exception{ValueError} is raised, and if it
+  does have a fragment identifier, \exception{ConfigurationError} is
+  raised.
+  This uses \method{isPath()} to determine whether \var{url-or-path}
+  is a URL of a filesystem path.
+\end{methoddesc}
+
+\begin{methoddesc}[loader]{openResource}{url}
+  Returns a resource object that represents the URL \var{url}.  The
+  URL is opened using the \function{urllib2.urlopen()} function, and
+  the returned resource object is created using
+  \method{createResource()}.  If the URL cannot be opened,
+  \exception{ConfigurationError} is raised.
+\end{methoddesc}
+
+\begin{methoddesc}[loader]{createResource}{file, url}
+  Returns a resource object for an open file and URL, given as
+  \var{file} and \var{url}, respectively.  This may be overridden by a
+  subclass if an alternate resource implementation is desired.
+\end{methoddesc}
+
+
+\section{\module{ZConfig.cmdline} --- Command-line override support}
+
+\declaremodule{}{ZConfig.cmdline}
+\modulesynopsis{Support for command-line overrides for configuration
+                settings.}
+
+This module exports an extended version of the \class{ConfigLoader}
+class from the \refmodule{ZConfig.loader} module.  This provides
+support for overriding specific settings from the configuration file
+from the command line, without requiring the application to provide
+specific options for everything the configuration file can include.
+
+\begin{classdesc}{ExtendedConfigLoader}{schema}
+  Construct a \class{ConfigLoader} subclass that adds support for
+  command-line overrides.
+\end{classdesc}
+
+The following additional method is provided, and is the only way to
+provide position information to associate with command-line
+parameters:
+
+\begin{methoddesc}{addOption}{spec\optional{, pos}}
+  Add a single value to the list of overridden values.  The \var{spec}
+  argument is a value specified, as described for the
+  \function{\refmodule{ZConfig}.loadConfig()} function.  A source
+  position for the specifier may be given as \var{pos}.  If \var{pos}
+  is specified and not \code{None}, it must be a sequence of three
+  values.  The first is the URL of the source (or some other
+  identifying string).  The second and third are the line number and
+  column of the setting.  These position information is only used to
+  construct a \exception{DataConversionError} when data conversion
+  fails.
+\end{methoddesc}
+
+
+\section{\module{ZConfig.substitution} --- String substitution}
+
+\declaremodule{}{ZConfig.substitution}
+\modulesynopsis{Shell-style string substitution helper.}
+
+This module provides a basic substitution facility similar to that
+found in the Bourne shell (\program{sh} on most \UNIX{} platforms).  
+
+The replacements supported by this module include:
+
+\begin{tableiii}{l|l|c}{code}{Source}{Replacement}{Notes}
+  \lineiii{\$\$}{\code{\$}}{(1)}
+  \lineiii{\$\var{name}}{The result of looking up \var{name}}{(2)}
+  \lineiii{\$\{\var{name}\}}{The result of looking up \var{name}}{}
+\end{tableiii}
+
+\noindent
+Notes:
+\begin{description}
+  \item[(1)]  This is different from the Bourne shell, which uses
+              \code{\textbackslash\$} to generate a \character{\$} in
+              the result text.  This difference avoids having as many
+              special characters in the syntax.
+
+  \item[(2)]  Any character which immediately follows \var{name} may
+              not be a valid character in a name.
+\end{description}
+
+In each case, \var{name} is a non-empty sequence of alphanumeric and
+underscore characters not starting with a digit.  If there is not a
+replacement for \var{name}, the exception
+\exception{SubstitutionReplacementError} is raised.
+Note that the lookup is expected to be case-insensitive; this module
+will always use a lower-case version of the name to perform the query.
+
+This module provides these functions:
+
+\begin{funcdesc}{substitute}{s, mapping}
+  Substitute values from \var{mapping} into \var{s}.  \var{mapping}
+  can be a \class{dict} or any type that supports the \method{get()}
+  method of the mapping protocol.  Replacement
+  values are copied into the result without further interpretation.
+  Raises \exception{SubstitutionSyntaxError} if there are malformed
+  constructs in \var{s}.
+\end{funcdesc}
+
+\begin{funcdesc}{isname}{s}
+  Returns \constant{True} if \var{s} is a valid name for a substitution
+  text, otherwise returns \constant{False}.
+\end{funcdesc}
+
+
+\subsection{Examples}
+
+\begin{verbatim}
+>>> from ZConfig.substitution import substitute
+>>> d = {'name': 'value',
+...      'top': '$middle',
+...      'middle' : 'bottom'}
+>>>
+>>> substitute('$name', d)
+'value'
+>>> substitute('$top', d)
+'$middle'
+\end{verbatim}
+
+
+\appendix
+\section{Schema Document Type Definition \label{schema-dtd}}
+
+The following is the XML Document Type Definition for \module{ZConfig}
+schema:
+
+\verbatiminput{schema.dtd}
+
+\end{document}

trunk/src/ZConfig/info.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Objects that can describe a ZConfig schema."""
+
+import copy
+
+import ZConfig
+
+
+class UnboundedThing:
+    __metaclass__ = type
+    __slots__ = ()
+
+    def __lt__(self, other):
+        return False
+
+    def __le__(self, other):
+        return isinstance(other, self.__class__)
+
+    def __gt__(self, other):
+        return True
+
+    def __ge__(self, other):
+        return True
+
+    def __eq__(self, other):
+        return isinstance(other, self.__class__)
+
+    def __ne__(self, other):
+        return not isinstance(other, self.__class__)
+
+    def __repr__(self):
+        return "<Unbounded>"
+
+Unbounded = UnboundedThing()
+
+
+class ValueInfo:
+    __metaclass__ = type
+    __slots__ = 'value', 'position'
+
+    def __init__(self, value, position):
+        self.value = value
+        # position is (lineno, colno, url)
+        self.position = position
+
+    def convert(self, datatype):
+        try:
+            return datatype(self.value)
+        except ValueError, e:
+            raise ZConfig.DataConversionError(e, self.value, self.position)
+
+
+class BaseInfo:
+    """Information about a single configuration key."""
+
+    description = None
+    example = None
+    metadefault = None
+
+    def __init__(self, name, datatype, minOccurs, maxOccurs, handler,
+                 attribute):
+        if maxOccurs is not None and maxOccurs < 1:
+            if maxOccurs < 1:
+                raise ZConfig.SchemaError(
+                    "maxOccurs must be at least 1")
+            if minOccurs is not None and minOccurs < maxOccurs:
+                raise ZConfig.SchemaError(
+                    "minOccurs must be at least maxOccurs")
+        self.name = name
+        self.datatype = datatype
+        self.minOccurs = minOccurs
+        self.maxOccurs = maxOccurs
+        self.handler = handler
+        self.attribute = attribute
+
+    def __repr__(self):
+        clsname = self.__class__.__name__
+        return "<%s for %s>" % (clsname, `self.name`)
+
+    def isabstract(self):
+        return False
+
+    def ismulti(self):
+        return self.maxOccurs > 1
+
+    def issection(self):
+        return False
+
+
+class BaseKeyInfo(BaseInfo):
+
+    _rawdefaults = None
+
+    def __init__(self, name, datatype, minOccurs, maxOccurs, handler,
+                 attribute):
+        assert minOccurs is not None
+        BaseInfo.__init__(self, name, datatype, minOccurs, maxOccurs,
+                          handler, attribute)
+        self._finished = False
+
+    def finish(self):
+        if self._finished:
+            raise ZConfig.SchemaError(
+                "cannot finish KeyInfo more than once")
+        self._finished = True
+
+    def adddefault(self, value, position, key=None):
+        if self._finished:
+            raise ZConfig.SchemaError(
+                "cannot add default values to finished KeyInfo")
+        # Check that the name/keyed relationship is right:
+        if self.name == "+" and key is None:
+            raise ZConfig.SchemaError(
+                "default values must be keyed for name='+'")
+        elif self.name != "+" and key is not None:
+            raise ZConfig.SchemaError(
+                "unexpected key for default value")
+        self.add_valueinfo(ValueInfo(value, position), key)
+
+    def add_valueinfo(self, vi, key):
+        """Actually add a ValueInfo to this key-info object.
+
+        The appropriate value of None-ness of key has already been
+        checked with regard to the name of the key, and has been found
+        permissible to add.
+
+        This method is a requirement for subclasses, and should not be
+        called by client code.
+        """
+        raise NotImplementedError(
+            "add_valueinfo() must be implemented by subclasses of BaseKeyInfo")
+
+    def prepare_raw_defaults(self):
+        assert self.name == "+"
+        if self._rawdefaults is None:
+            self._rawdefaults = self._default
+        self._default = {}
+
+
+class KeyInfo(BaseKeyInfo):
+
+    _default = None
+
+    def __init__(self, name, datatype, minOccurs, handler, attribute):
+        BaseKeyInfo.__init__(self, name, datatype, minOccurs, 1,
+                             handler, attribute)
+        if self.name == "+":
+            self._default = {}
+
+    def add_valueinfo(self, vi, key):
+        if self.name == "+":
+            if self._default.has_key(key):
+                # not ideal: we're presenting the unconverted
+                # version of the key
+                raise ZConfig.SchemaError(
+                    "duplicate default value for key %s" % `key`)
+            self._default[key] = vi
+        elif self._default is not None:
+            raise ZConfig.SchemaError(
+                "cannot set more than one default to key with maxOccurs == 1")
+        else:
+            self._default = vi
+
+    def computedefault(self, keytype):
+        self.prepare_raw_defaults()
+        for k, vi in self._rawdefaults.iteritems():
+            key = ValueInfo(k, vi.position).convert(keytype)
+            self.add_valueinfo(vi, key)
+
+    def getdefault(self):
+        # Use copy.copy() to make sure we don't allow polution of
+        # our internal data without having to worry about both the
+        # list and dictionary cases:
+        return copy.copy(self._default)
+
+
+class MultiKeyInfo(BaseKeyInfo):
+
+    def __init__(self, name, datatype, minOccurs, maxOccurs, handler,
+                 attribute):
+        BaseKeyInfo.__init__(self, name, datatype, minOccurs, maxOccurs,
+                             handler, attribute)
+        if self.name == "+":
+            self._default = {}
+        else:
+            self._default = []
+
+    def add_valueinfo(self, vi, key):
+        if self.name == "+":
+            # This is a keyed value, not a simple value:
+            if key in self._default:
+                self._default[key].append(vi)
+            else:
+                self._default[key] = [vi]
+        else:
+            self._default.append(vi)
+
+    def computedefault(self, keytype):
+        self.prepare_raw_defaults()
+        for k, vlist in self._rawdefaults.iteritems():
+            key = ValueInfo(k, vlist[0].position).convert(keytype)
+            for vi in vlist:
+                self.add_valueinfo(vi, key)
+
+    def getdefault(self):
+        return copy.copy(self._default)
+
+
+class SectionInfo(BaseInfo):
+    def __init__(self, name, sectiontype, minOccurs, maxOccurs, handler,
+                 attribute):
+        # name        - name of the section; one of '*', '+', or name1
+        # sectiontype - SectionType instance
+        # minOccurs   - minimum number of occurances of the section
+        # maxOccurs   - maximum number of occurances; if > 1, name
+        #               must be '*' or '+'
+        # handler     - handler name called when value(s) must take effect,
+        #               or None
+        # attribute   - name of the attribute on the SectionValue object
+        if maxOccurs > 1:
+            if name not in ('*', '+'):
+                raise ZConfig.SchemaError(
+                    "sections which can occur more than once must"
+                    " use a name of '*' or '+'")
+            if not attribute:
+                raise ZConfig.SchemaError(
+                    "sections which can occur more than once must"
+                    " specify a target attribute name")
+        if sectiontype.isabstract():
+            datatype = None
+        else:
+            datatype = sectiontype.datatype
+        BaseInfo.__init__(self, name, datatype,
+                          minOccurs, maxOccurs, handler, attribute)
+        self.sectiontype = sectiontype
+
+    def __repr__(self):
+        clsname = self.__class__.__name__
+        return "<%s for %s (%s)>" % (
+            clsname, self.sectiontype.name, `self.name`)
+
+    def issection(self):
+        return True
+
+    def allowUnnamed(self):
+        return self.name == "*"
+
+    def isAllowedName(self, name):
+        if name == "*" or name == "+":
+            return False
+        elif self.name == "+":
+            return name and True or False
+        elif self.name == "*":
+            return True
+        else:
+            return name == self.name
+
+    def getdefault(self):
+        # sections cannot have defaults
+        if self.maxOccurs > 1:
+            return []
+        else:
+            return None
+
+
+class AbstractType:
+    __metaclass__ = type
+    __slots__ = '_subtypes', 'name', 'description'
+
+    def __init__(self, name):
+        self._subtypes = {}
+        self.name = name
+        self.description = None
+
+    def addsubtype(self, type):
+        self._subtypes[type.name] = type
+
+    def getsubtype(self, name):
+        try:
+            return self._subtypes[name]
+        except KeyError:
+            raise ZConfig.SchemaError("no sectiontype %s in abstracttype %s"
+                                      % (`name`, `self.name`))
+
+    def hassubtype(self, name):
+        """Return true iff this type has 'name' as a concrete manifestation."""
+        return name in self._subtypes.keys()
+
+    def getsubtypenames(self):
+        """Return the names of all concrete types as a sorted list."""
+        L = self._subtypes.keys()
+        L.sort()
+        return L
+
+    def isabstract(self):
+        return True
+
+
+class SectionType:
+    def __init__(self, name, keytype, valuetype, datatype, registry, types):
+        # name      - name of the section, or '*' or '+'
+        # datatype  - type for the section itself
+        # keytype   - type for the keys themselves
+        # valuetype - default type for key values
+        self.name = name
+        self.datatype = datatype
+        self.keytype = keytype
+        self.valuetype = valuetype
+        self.handler = None
+        self.description = None
+        self.registry = registry
+        self._children = []    # [(key, info), ...]
+        self._attrmap = {}     # {attribute: info, ...}
+        self._keymap = {}      # {key: info, ...}
+        self._types = types
+
+    def gettype(self, name):
+        n = name.lower()
+        try:
+            return self._types[n]
+        except KeyError:
+            raise ZConfig.SchemaError("unknown type name: " + `name`)
+
+    def gettypenames(self):
+        return self._types.keys()
+
+    def __len__(self):
+        return len(self._children)
+
+    def __getitem__(self, index):
+        return self._children[index]
+
+    def _add_child(self, key, info):
+        # check naming constraints
+        assert key or info.attribute
+        if key and self._keymap.has_key(key):
+            raise ZConfig.SchemaError(
+                "child name %s already used" % key)
+        if info.attribute and self._attrmap.has_key(info.attribute):
+            raise ZConfig.SchemaError(
+                "child attribute name %s already used" % info.attribute)
+        # a-ok, add the item to the appropriate maps
+        if info.attribute:
+            self._attrmap[info.attribute] = info
+        if key:
+            self._keymap[key] = info
+        self._children.append((key, info))
+
+    def addkey(self, keyinfo):
+        self._add_child(keyinfo.name, keyinfo)
+
+    def addsection(self, name, sectinfo):
+        assert name not in ("*", "+")
+        self._add_child(name, sectinfo)
+
+    def getinfo(self, key):
+        if not key:
+            raise ZConfig.ConfigurationError(
+                "cannot match a key without a name")
+        try:
+            return self._keymap[key]
+        except KeyError:
+            raise ZConfig.ConfigurationError("no key matching " + `key`)
+
+    def getrequiredtypes(self):
+        d = {}
+        if self.name:
+            d[self.name] = 1
+        stack = [self]
+        while stack:
+            info = stack.pop()
+            for key, ci in info._children:
+                if ci.issection():
+                    t = ci.sectiontype
+                    if not d.has_key(t.name):
+                        d[t.name] = 1
+                        stack.append(t)
+        return d.keys()
+
+    def getsectioninfo(self, type, name):
+        for key, info in self._children:
+            if key:
+                if key == name:
+                    if not info.issection():
+                        raise ZConfig.ConfigurationError(
+                            "section name %s already in use for key" % key)
+                    st = info.sectiontype
+                    if st.isabstract():
+                        try:
+                            st = st.getsubtype(type)
+                        except ZConfig.ConfigurationError:
+                            raise ZConfig.ConfigurationError(
+                                "section type %s not allowed for name %s"
+                                % (`type`, `key`))
+                    if not st.name == type:
+                        raise ZConfig.ConfigurationError(
+                            "name %s must be used for a %s section"
+                            % (`name`, `st.name`))
+                    return info
+            # else must be a sectiontype or an abstracttype:
+            elif info.sectiontype.name == type:
+                if not (name or info.allowUnnamed()):
+                    raise ZConfig.ConfigurationError(
+                        `type` + " sections must be named")
+                return info
+            elif info.sectiontype.isabstract():
+                st = info.sectiontype
+                if st.name == type:
+                    raise ZConfig.ConfigurationError(
+                        "cannot define section with an abstract type")
+                try:
+                    st = st.getsubtype(type)
+                except ZConfig.ConfigurationError:
+                    # not this one; maybe a different one
+                    pass
+                else:
+                    return info
+        raise ZConfig.ConfigurationError(
+            "no matching section defined for type='%s', name='%s'" % (
+            type, name))
+
+    def isabstract(self):
+        return False
+
+
+class SchemaType(SectionType):
+    def __init__(self, keytype, valuetype, datatype, handler, url,
+                 registry):
+        SectionType.__init__(self, None, keytype, valuetype, datatype,
+                             registry, {})
+        self._components = {}
+        self.handler = handler
+        self.url = url
+
+    def addtype(self, typeinfo):
+        n = typeinfo.name
+        if self._types.has_key(n):
+            raise ZConfig.SchemaError("type name cannot be redefined: "
+                                      + `typeinfo.name`)
+        self._types[n] = typeinfo
+
+    def allowUnnamed(self):
+        return True
+
+    def isAllowedName(self, name):
+        return False
+
+    def issection(self):
+        return True
+
+    def getunusedtypes(self):
+        alltypes = self.gettypenames()
+        reqtypes = self.getrequiredtypes()
+        for n in reqtypes:
+            alltypes.remove(n)
+        if self.name and self.name in alltypes:
+            alltypes.remove(self.name)
+        return alltypes
+
+    def createSectionType(self, name, keytype, valuetype, datatype):
+        t = SectionType(name, keytype, valuetype, datatype,
+                        self.registry, self._types)
+        self.addtype(t)
+        return t
+
+    def deriveSectionType(self, base, name, keytype, valuetype, datatype):
+        if isinstance(base, SchemaType):
+            raise ZConfig.SchemaError(
+                "cannot derive sectiontype from top-level schema")
+        t = self.createSectionType(name, keytype, valuetype, datatype)
+        t._attrmap.update(base._attrmap)
+        t._keymap.update(base._keymap)
+        t._children.extend(base._children)
+        for i in range(len(t._children)):
+            key, info = t._children[i]
+            if isinstance(info, BaseKeyInfo) and info.name == "+":
+                # need to create a new info object and recompute the
+                # default mapping based on the new keytype
+                info = copy.copy(info)
+                info.computedefault(t.keytype)
+                t._children[i] = (key, info)
+        return t
+
+    def addComponent(self, name):
+        if self._components.has_key(name):
+            raise ZConfig.SchemaError("already have component %s" % name)
+        self._components[name] = name
+
+    def hasComponent(self, name):
+        return self._components.has_key(name)
+
+
+def createDerivedSchema(base):
+    new = SchemaType(base.keytype, base.valuetype, base.datatype,
+                     base.handler, base.url, base.registry)
+    new._components.update(base._components)
+    new.description = base.description
+    new._children[:] = base._children
+    new._attrmap.update(base._attrmap)
+    new._keymap.update(base._keymap)
+    new._types.update(base._types)
+    return new

trunk/src/ZConfig/loader.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Schema loader utility."""
+
+import os.path
+import sys
+import urllib
+import urllib2
+
+import ZConfig
+import ZConfig.cfgparser
+import ZConfig.datatypes
+import ZConfig.info
+import ZConfig.matcher
+import ZConfig.schema
+import ZConfig.url
+
+
+def loadSchema(url):
+    return SchemaLoader().loadURL(url)
+
+def loadSchemaFile(file, url=None):
+    return SchemaLoader().loadFile(file, url)
+
+def loadConfig(schema, url, overrides=()):
+    return _get_config_loader(schema, overrides).loadURL(url)
+
+def loadConfigFile(schema, file, url=None, overrides=()):
+    return _get_config_loader(schema, overrides).loadFile(file, url)
+
+
+def _get_config_loader(schema, overrides):
+    if overrides:
+        from ZConfig import cmdline
+        loader = cmdline.ExtendedConfigLoader(schema)
+        for opt in overrides:
+            loader.addOption(opt)
+    else:
+        loader = ConfigLoader(schema)
+    return loader
+
+
+class BaseLoader:
+    def __init__(self):
+        pass
+
+    def createResource(self, file, url):
+        return Resource(file, url)
+
+    def loadURL(self, url):
+        url = self.normalizeURL(url)
+        r = self.openResource(url)
+        try:
+            return self.loadResource(r)
+        finally:
+            r.close()
+
+    def loadFile(self, file, url=None):
+        if not url:
+            url = _url_from_file(file)
+        r = self.createResource(file, url)
+        try:
+            return self.loadResource(r)
+        finally:
+            r.close()
+
+    # utilities
+
+    def loadResource(self, resource):
+        raise NotImplementedError(
+            "BaseLoader.loadResource() must be overridden by a subclass")
+
+    def openResource(self, url):
+        # ConfigurationError exceptions raised here should be
+        # str()able to generate a message for an end user.
+        #
+        # XXX This should be replaced to use a local cache for remote
+        # resources.  The policy needs to support both re-retrieve on
+        # change and provide the cached resource when the remote
+        # resource is not accessible.
+        url = str(url)
+        try:
+            file = urllib2.urlopen(url)
+        except urllib2.URLError, e:
+            # urllib2.URLError has a particularly hostile str(), so we
+            # generally don't want to pass it along to the user.
+            self._raise_open_error(url, e.reason)
+        except (IOError, OSError), e:
+            # Python 2.1 raises a different error from Python 2.2+,
+            # so we catch both to make sure we detect the situation.
+            self._raise_open_error(url, str(e))
+        return self.createResource(file, url)
+
+    def _raise_open_error(self, url, message):
+        if url[:7].lower() == "file://":
+            what = "file"
+            ident = urllib.url2pathname(url[7:])
+        else:
+            what = "URL"
+            ident = url
+        raise ZConfig.ConfigurationError(
+            "error opening %s %s: %s" % (what, ident, message),
+            url)
+
+    def normalizeURL(self, url):
+        if self.isPath(url):
+            url = "file://" + urllib.pathname2url(os.path.abspath(url))
+        newurl, fragment = ZConfig.url.urldefrag(url)
+        if fragment:
+            raise ZConfig.ConfigurationError(
+                "fragment identifiers are not supported",
+                url)
+        return newurl
+
+    def isPath(self, s):
+        """Return True iff 's' should be handled as a filesystem path."""
+        if ":" in s:
+            # XXX This assumes that one-character scheme identifiers
+            # are always Windows drive letters; I don't know of any
+            # one-character scheme identifiers.
+            scheme, rest = urllib.splittype(s)
+            return len(scheme) == 1
+        else:
+            return True
+
+
+
+def _url_from_file(file):
+    name = getattr(file, "name", None)
+    if name and name[0] != "<" and name[-1] != ">":
+        return "file://" + urllib.pathname2url(os.path.abspath(name))
+    else:
+        return None
+
+
+class SchemaLoader(BaseLoader):
+    def __init__(self, registry=None):
+        if registry is None:
+            registry = ZConfig.datatypes.Registry()
+        BaseLoader.__init__(self)
+        self.registry = registry
+        self._cache = {}
+
+    def loadResource(self, resource):
+        if resource.url and self._cache.has_key(resource.url):
+            schema = self._cache[resource.url]
+        else:
+            schema = ZConfig.schema.parseResource(resource, self)
+            self._cache[resource.url] = schema
+        return schema
+
+    # schema parser support API
+
+    def schemaComponentSource(self, package, file):
+        parts = package.split(".")
+        if not parts:
+            raise ZConfig.SchemaError(
+                "illegal schema component name: " + `package`)
+        if "" in parts:
+            # '' somewhere in the package spec; still illegal
+            raise ZConfig.SchemaError(
+                "illegal schema component name: " + `package`)
+        file = file or "component.xml"
+        try:
+            __import__(package)
+        except ImportError, e:
+            raise ZConfig.SchemaResourceError(
+                "could not load package %s: %s" % (package, str(e)),
+                filename=file,
+                package=package)
+        pkg = sys.modules[package]
+        if not hasattr(pkg, "__path__"):
+            raise ZConfig.SchemaResourceError(
+                "import name does not refer to a package",
+                filename=file, package=package)
+        for dir in pkg.__path__:
+            dirname = os.path.abspath(dir)
+            fn = os.path.join(dirname, file)
+            if os.path.exists(fn):
+                return "file://" + urllib.pathname2url(fn)
+        else:
+            raise ZConfig.SchemaResourceError("schema component not found",
+                                              filename=file,
+                                              package=package,
+                                              path=pkg.__path__)
+
+
+class ConfigLoader(BaseLoader):
+    def __init__(self, schema):
+        if schema.isabstract():
+            raise ZConfig.SchemaError(
+                "cannot check a configuration an abstract type")
+        BaseLoader.__init__(self)
+        self.schema = schema
+        self._private_schema = False
+
+    def loadResource(self, resource):
+        sm = self.createSchemaMatcher()
+        self._parse_resource(sm, resource)
+        result = sm.finish(), CompositeHandler(sm.handlers, self.schema)
+        return result
+
+    def createSchemaMatcher(self):
+        return ZConfig.matcher.SchemaMatcher(self.schema)
+
+    # config parser support API
+
+    def startSection(self, parent, type, name):
+        t = self.schema.gettype(type)
+        if t.isabstract():
+            raise ZConfig.ConfigurationError(
+                "concrete sections cannot match abstract section types;"
+                " found abstract type " + `type`)
+        return parent.createChildMatcher(t, name)
+
+    def endSection(self, parent, type, name, matcher):
+        sectvalue = matcher.finish()
+        parent.addSection(type, name, sectvalue)
+
+    def importSchemaComponent(self, pkgname):
+        schema = self.schema
+        if not self._private_schema:
+            # replace the schema with an extended schema on the first %import
+            self._loader = SchemaLoader(self.schema.registry)
+            schema = ZConfig.info.createDerivedSchema(self.schema)
+            self._private_schema = True
+            self.schema = schema
+        url = self._loader.schemaComponentSource(pkgname, '')
+        if schema.hasComponent(url):
+            return
+        resource = self.openResource(url)
+        schema.addComponent(url)
+        try:
+            ZConfig.schema.parseComponent(resource, self._loader, schema)
+        finally:
+            resource.close()
+
+    def includeConfiguration(self, section, url, defines):
+        url = self.normalizeURL(url)
+        r = self.openResource(url)
+        try:
+            self._parse_resource(section, r, defines)
+        finally:
+            r.close()
+
+    # internal helper
+
+    def _parse_resource(self, matcher, resource, defines=None):
+        parser = ZConfig.cfgparser.ZConfigParser(resource, self, defines)
+        parser.parse(matcher)
+
+
+class CompositeHandler:
+
+    def __init__(self, handlers, schema):
+        self._handlers = handlers
+        self._convert = schema.registry.get("basic-key")
+
+    def __call__(self, handlermap):
+        d = {}
+        for name, callback in handlermap.items():
+            n = self._convert(name)
+            if d.has_key(n):
+                raise ZConfig.ConfigurationError(
+                    "handler name not unique when converted to a basic-key: "
+                    + `name`)
+            d[n] = callback
+        L = []
+        for handler, value in self._handlers:
+            if not d.has_key(handler):
+                L.append(handler)
+        if L:
+            raise ZConfig.ConfigurationError(
+                "undefined handlers: " + ", ".join(L))
+        for handler, value in self._handlers:
+            f = d[handler]
+            if f is not None:
+                f(value)
+
+    def __len__(self):
+        return len(self._handlers)
+
+
+class Resource:
+    def __init__(self, file, url):
+        self.file = file
+        self.url = url
+
+    def close(self):
+        if self.file is not None:
+            self.file.close()
+            self.file = None
+            self.closed = True
+
+    def __getattr__(self, name):
+        return getattr(self.file, name)

trunk/src/ZConfig/matcher.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Utility that manages the binding of configuration data to a section."""
+
+import ZConfig
+
+from ZConfig.info import ValueInfo
+
+
+class BaseMatcher:
+    def __init__(self, info, type, handlers):
+        self.info = info
+        self.type = type
+        self._values = {}
+        for key, info in type:
+            if info.name == "+" and not info.issection():
+                v = {}
+            elif info.ismulti():
+                v = []
+            else:
+                v = None
+            assert info.attribute is not None
+            self._values[info.attribute] = v
+        self._sectionnames = {}
+        if handlers is None:
+            handlers = []
+        self.handlers = handlers
+
+    def __repr__(self):
+        clsname = self.__class__.__name__
+        extra = "type " + `self.type.name`
+        return "<%s for %s>" % (clsname, extra)
+
+    def addSection(self, type, name, sectvalue):
+        if name:
+            if self._sectionnames.has_key(name):
+                raise ZConfig.ConfigurationError(
+                    "section names must not be re-used within the"
+                    " same container:" + `name`)
+            self._sectionnames[name] = name
+        ci = self.type.getsectioninfo(type, name)
+        attr = ci.attribute
+        v = self._values[attr]
+        if ci.ismulti():
+            v.append(sectvalue)
+        elif v is None:
+            self._values[attr] = sectvalue
+        else:
+            raise ZConfig.ConfigurationError(
+                "too many instances of %s section" % `ci.sectiontype.name`)
+
+    def addValue(self, key, value, position):
+        try:
+            realkey = self.type.keytype(key)
+        except ValueError, e:
+            raise ZConfig.DataConversionError(e, key, position)
+        arbkey_info = None
+        for i in range(len(self.type)):
+            k, ci = self.type[i]
+            if k == realkey:
+                break
+            if ci.name == "+" and not ci.issection():
+                arbkey_info = k, ci
+        else:
+            if arbkey_info is None:
+                raise ZConfig.ConfigurationError(
+                    `key` + " is not a known key name")
+            k, ci = arbkey_info
+        if ci.issection():
+            if ci.name:
+                extra = " in %s sections" % `self.type.name`
+            else:
+                extra = ""
+            raise ZConfig.ConfigurationError(
+                "%s is not a valid key name%s" % (`key`, extra))
+
+        ismulti = ci.ismulti()
+        attr = ci.attribute
+        assert attr is not None
+        v = self._values[attr]
+        if v is None:
+            if k == '+':
+                v = {}
+            elif ismulti:
+                v = []
+            self._values[attr] = v
+        elif not ismulti:
+            if k != '+':
+                raise ZConfig.ConfigurationError(
+                    `key` + " does not support multiple values")
+        elif len(v) == ci.maxOccurs:
+            raise ZConfig.ConfigurationError(
+                "too many values for " + `name`)
+
+        value = ValueInfo(value, position)
+        if k == '+':
+            if ismulti:
+                if v.has_key(realkey):
+                    v[realkey].append(value)
+                else:
+                    v[realkey] = [value]
+            else:
+                if v.has_key(realkey):
+                    raise ZConfig.ConfigurationError(
+                        "too many values for " + `key`)
+                v[realkey] = value
+        elif ismulti:
+            v.append(value)
+        else:
+            self._values[attr] = value
+
+    def createChildMatcher(self, type, name):
+        ci = self.type.getsectioninfo(type.name, name)
+        assert not ci.isabstract()
+        if not ci.isAllowedName(name):
+            raise ZConfig.ConfigurationError(
+                "%s is not an allowed name for %s sections"
+                % (`name`, `ci.sectiontype.name`))
+        return SectionMatcher(ci, type, name, self.handlers)
+
+    def finish(self):
+        """Check the constraints of the section and convert to an application
+        object."""
+        values = self._values
+        for key, ci in self.type:
+            if key:
+                key = repr(key)
+            else:
+                key = "section type " + `ci.sectiontype.name`
+            assert ci.attribute is not None
+            attr = ci.attribute
+            v = values[attr]
+            if ci.name == '+' and not ci.issection():
+                # v is a dict
+                if ci.minOccurs > len(v):
+                    raise ZConfig.ConfigurationError(
+                        "no keys defined for the %s key/value map; at least %d"
+                        " must be specified" % (attr, ci.minOccurs))
+            if v is None and ci.minOccurs:
+                default = ci.getdefault()
+                if default is None:
+                    raise ZConfig.ConfigurationError(
+                        "no values for %s; %s required" % (key, ci.minOccurs))
+                else:
+                    v = values[attr] = default[:]
+            if ci.ismulti():
+                if not v:
+                    default = ci.getdefault()
+                    if isinstance(default, dict):
+                        v.update(default)
+                    else:
+                        v[:] = default
+                if len(v) < ci.minOccurs:
+                    raise ZConfig.ConfigurationError(
+                        "not enough values for %s; %d found, %d required"
+                        % (key, len(v), ci.minOccurs))
+            if v is None and not ci.issection():
+                if ci.ismulti():
+                    v = ci.getdefault()[:]
+                else:
+                    v = ci.getdefault()
+                values[attr] = v
+        return self.constuct()
+
+    def constuct(self):
+        values = self._values
+        for name, ci in self.type:
+            assert ci.attribute is not None
+            attr = ci.attribute
+            if ci.ismulti():
+                if ci.issection():
+                    v = []
+                    for s in values[attr]:
+                        if s is not None:
+                            st = s.getSectionDefinition()
+                            try:
+                                s = st.datatype(s)
+                            except ValueError, e:
+                                raise ZConfig.DataConversionError(
+                                    e, s, (-1, -1, None))
+                        v.append(s)
+                elif ci.name == '+':
+                    v = values[attr]
+                    for key, val in v.items():
+                        v[key] = [vi.convert(ci.datatype) for vi in val]
+                else:
+                    v = [vi.convert(ci.datatype) for vi in values[attr]]
+            elif ci.issection():
+                if values[attr] is not None:
+                    st = values[attr].getSectionDefinition()
+                    try:
+                        v = st.datatype(values[attr])
+                    except ValueError, e:
+                        raise ZConfig.DataConversionError(
+                            e, values[attr], (-1, -1, None))
+                else:
+                    v = None
+            elif name == '+':
+                v = values[attr]
+                if not v:
+                    for key, val in ci.getdefault().items():
+                        v[key] = val.convert(ci.datatype)
+                else:
+                    for key, val in v.items():
+                        v[key] = val.convert(ci.datatype)
+            else:
+                v = values[attr]
+                if v is not None:
+                    v = v.convert(ci.datatype)
+            values[attr] = v
+            if ci.handler is not None:
+                self.handlers.append((ci.handler, v))
+        return self.createValue()
+
+    def createValue(self):
+        return SectionValue(self._values, None, self)
+
+
+class SectionMatcher(BaseMatcher):
+    def __init__(self, info, type, name, handlers):
+        if name or info.allowUnnamed():
+            self.name = name
+        else:
+            raise ZConfig.ConfigurationError(
+                `type.name` + " sections may not be unnamed")
+        BaseMatcher.__init__(self, info, type, handlers)
+
+    def createValue(self):
+        return SectionValue(self._values, self.name, self)
+
+
+class SchemaMatcher(BaseMatcher):
+    def __init__(self, schema):
+        BaseMatcher.__init__(self, schema, schema, [])
+
+    def finish(self):
+        # Since there's no outer container to call datatype()
+        # for the schema, we convert on the way out.
+        v = BaseMatcher.finish(self)
+        v = self.type.datatype(v)
+        if self.type.handler is not None:
+            self.handlers.append((self.type.handler, v))
+        return v
+
+
+class SectionValue:
+    """Generic 'bag-of-values' object for a section.
+
+    Derived classes should always call the SectionValue constructor
+    before attempting to modify self.
+    """
+
+    def __init__(self, values, name, matcher):
+        self.__dict__.update(values)
+        self._name = name
+        self._matcher = matcher
+        self._attributes = tuple(values.keys())
+
+    def __repr__(self):
+        if self._name:
+            # probably unique for a given config file; more readable than id()
+            name = `self._name`
+        else:
+            # identify uniquely
+            name = "at %#x" % id(self)
+        clsname = self.__class__.__name__
+        return "<%s for %s %s>" % (clsname, self._matcher.type.name, name)
+
+    def __str__(self):
+        l = []
+        attrnames = [s for s in self.__dict__.keys() if s[0] != "_"]
+        attrnames.sort()
+        for k in attrnames:
+            v = getattr(self, k)
+            l.append('%-40s: %s' % (k, v))
+        return '\n'.join(l)
+
+    def getSectionName(self):
+        return self._name
+
+    def getSectionType(self):
+        return self._matcher.type.name
+
+    def getSectionDefinition(self):
+        return self._matcher.type
+
+    def getSectionMatcher(self):
+        return self._matcher
+
+    def getSectionAttributes(self):
+        return self._attributes

trunk/src/ZConfig/schema.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Parser for ZConfig schemas."""
+
+import os
+import xml.sax
+
+import ZConfig
+
+from ZConfig import info
+from ZConfig import url
+
+
+def parseResource(resource, loader):
+    parser = SchemaParser(loader, resource.url)
+    xml.sax.parse(resource.file, parser)
+    return parser._schema
+
+
+def parseComponent(resource, loader, schema):
+    parser = ComponentParser(loader, resource.url, schema)
+    xml.sax.parse(resource.file, parser)
+
+
+def _srepr(ob):
+    if isinstance(ob, type(u'')):
+        # drop the leading "u" from a unicode repr
+        return `ob`[1:]
+    else:
+        return `ob`
+
+
+class BaseParser(xml.sax.ContentHandler):
+
+    _cdata_tags = "description", "metadefault", "example", "default"
+    _handled_tags = ("import", "abstracttype", "sectiontype",
+                     "key", "multikey", "section", "multisection")
+
+    _allowed_parents = {
+        "description": ["key", "section", "multikey", "multisection",
+                        "sectiontype", "abstracttype",
+                        "schema", "component"],
+        "example": ["key", "section", "multikey", "multisection"],
+        "metadefault": ["key", "section", "multikey", "multisection"],
+        "default": ["key", "multikey"],
+        "import": ["schema", "component"],
+        "abstracttype": ["schema", "component"],
+        "sectiontype": ["schema", "component"],
+        "key": ["schema", "sectiontype"],
+        "multikey": ["schema", "sectiontype"],
+        "section": ["schema", "sectiontype"],
+        "multisection": ["schema", "sectiontype"],
+        }
+
+    def __init__(self, loader, url):
+        self._registry = loader.registry
+        self._loader = loader
+        self._basic_key = self._registry.get("basic-key")
+        self._identifier = self._registry.get("identifier")
+        self._cdata = None
+        self._locator = None
+        self._prefixes = []
+        self._schema = None
+        self._stack = []
+        self._url = url
+        self._elem_stack = []
+
+    # SAX 2 ContentHandler methods
+
+    def setDocumentLocator(self, locator):
+        self._locator = locator
+
+    def startElement(self, name, attrs):
+        attrs = dict(attrs)
+        if self._elem_stack:
+            parent = self._elem_stack[-1]
+            if not self._allowed_parents.has_key(name):
+                self.error("Unknown tag " + name)
+            if parent not in self._allowed_parents[name]:
+                self.error("%s elements may not be nested in %s elements"
+                           % (_srepr(name), _srepr(parent)))
+        elif name != self._top_level:
+            self.error("Unknown document type " + name)
+        self._elem_stack.append(name)
+        if name == self._top_level:
+            if self._schema is not None:
+                self.error("schema element improperly nested")
+            getattr(self, "start_" + name)(attrs)
+        elif name in self._handled_tags:
+            if self._schema is None:
+                self.error(name + " element outside of schema")
+            getattr(self, "start_" + name)(attrs)
+        elif name in self._cdata_tags:
+            if self._schema is None:
+                self.error(name + " element outside of schema")
+            if self._cdata is not None:
+                self.error(name + " element improperly nested")
+            self._cdata = []
+            self._position = None
+            self._attrs = attrs
+
+    def characters(self, data):
+        if self._cdata is not None:
+            if self._position is None:
+                self._position = self.get_position()
+            self._cdata.append(data)
+        elif data.strip():
+            self.error("unexpected non-blank character data: "
+                       + `data.strip()`)
+
+    def endElement(self, name):
+        del self._elem_stack[-1]
+        if name in self._handled_tags:
+            getattr(self, "end_" + name)()
+        else:
+            data = ''.join(self._cdata).strip()
+            self._cdata = None
+            getattr(self, "characters_" + name)(data)
+
+    def endDocument(self):
+        if self._schema is None:
+            self.error("no %s found" % self._top_level)
+
+    # helper methods
+
+    def get_position(self):
+        if self._locator:
+            return (self._locator.getLineNumber(),
+                    self._locator.getColumnNumber(),
+                    (self._locator.getSystemId() or self._url))
+        else:
+            return None, None, self._url
+
+    def get_handler(self, attrs):
+        v = attrs.get("handler")
+        if v is None:
+            return v
+        else:
+            return self.basic_key(v)
+
+    def push_prefix(self, attrs):
+        name = attrs.get("prefix")
+        if name:
+            if self._prefixes:
+                convert = self._registry.get("dotted-suffix")
+            else:
+                convert = self._registry.get("dotted-name")
+            try:
+                name = convert(name)
+            except ValueError, err:
+                self.error("not a valid prefix: %s (%s)"
+                           % (_srepr(name), str(err)))
+            if name[0] == ".":
+                prefix = self._prefixes[-1] + name
+            else:
+                prefix = name
+        elif self._prefixes:
+            prefix = self._prefixes[-1]
+        else:
+            prefix = ''
+        self._prefixes.append(prefix)
+
+    def pop_prefix(self):
+        del self._prefixes[-1]
+
+    def get_classname(self, name):
+        name = str(name)
+        if name.startswith("."):
+            return self._prefixes[-1] + name
+        else:
+            return name
+
+    def get_datatype(self, attrs, attrkey, default, base=None):
+        if attrs.has_key(attrkey):
+            dtname = self.get_classname(attrs[attrkey])
+        else:
+            convert = getattr(base, attrkey, None)
+            if convert is not None:
+                return convert
+            dtname = default
+
+        try:
+            return self._registry.get(dtname)
+        except ValueError, e:
+            self.error(e[0])
+
+    def get_sect_typeinfo(self, attrs, base=None):
+        keytype = self.get_datatype(attrs, "keytype", "basic-key", base)
+        valuetype = self.get_datatype(attrs, "valuetype", "string")
+        datatype = self.get_datatype(attrs, "datatype", "null", base)
+        return keytype, valuetype, datatype
+
+    def get_required(self, attrs):
+        if attrs.has_key("required"):
+            v = attrs["required"]
+            if v == "yes":
+                return True
+            elif v == "no":
+                return False
+            self.error("value for 'required' must be 'yes' or 'no'")
+        else:
+            return False
+
+    def get_ordinality(self, attrs):
+        min, max = 0, info.Unbounded
+        if self.get_required(attrs):
+            min = 1
+        return min, max
+
+    def get_sectiontype(self, attrs):
+        type = attrs.get("type")
+        if not type:
+            self.error("section must specify type")
+        return self._schema.gettype(type)
+
+    def get_key_info(self, attrs, element):
+        any, name, attribute = self.get_name_info(attrs, element)
+        if any == '*':
+            self.error(element + " may not specify '*' for name")
+        if not name and any != '+':
+            self.error(element + " name may not be omitted or empty")
+        datatype = self.get_datatype(attrs, "datatype", "string")
+        handler = self.get_handler(attrs)
+        return name or any, datatype, handler, attribute
+
+    def get_name_info(self, attrs, element):
+        name = attrs.get("name")
+        if not name:
+            self.error(element + " name must be specified and non-empty")
+        aname = attrs.get("attribute")
+        if aname:
+            aname = self.identifier(aname)
+            if aname.startswith("getSection"):
+                # reserved; used for SectionValue methods to get meta-info
+                self.error("attribute names may not start with 'getSection'")
+        if name in ("*", "+"):
+            if not aname:
+                self.error(
+                    "container attribute must be specified and non-empty"
+                    " when using '*' or '+' for a section name")
+            return name, None, aname
+        else:
+            # run the keytype converter to make sure this is a valid key
+            try:
+                name = self._stack[-1].keytype(name)
+            except ValueError, e:
+                self.error("could not convert key name to keytype: " + str(e))
+            if not aname:
+                aname = self.basic_key(name)
+                aname = self.identifier(aname.replace('-', '_'))
+            return None, name, aname
+
+    # schema loading logic
+
+    def characters_default(self, data):
+        key = self._attrs.get("key")
+        self._stack[-1].adddefault(data, self._position, key)
+
+    def characters_description(self, data):
+        if self._stack[-1].description is not None:
+            self.error(
+                "at most one <description> may be used for each element")
+        self._stack[-1].description = data
+
+    def characters_example(self, data):
+        self._stack[-1].example = data
+
+    def characters_metadefault(self, data):
+        self._stack[-1].metadefault = data
+
+    def start_import(self, attrs):
+        src = attrs.get("src", "").strip()
+        pkg = attrs.get("package", "").strip()
+        file = attrs.get("file", "").strip()
+        if not (src or pkg):
+            self.error("import must specify either src or package")
+        if src and pkg:
+            self.error("import may only specify one of src or package")
+        if src:
+            if file:
+                self.error("import may not specify file and src")
+            src = url.urljoin(self._url, src)
+            src, fragment = url.urldefrag(src)
+            if fragment:
+                self.error("import src many not include"
+                           " a fragment identifier")
+            schema = self._loader.loadURL(src)
+            for n in schema.gettypenames():
+                self._schema.addtype(schema.gettype(n))
+        else:
+            if os.path.dirname(file):
+                self.error("file may not include a directory part")
+            pkg = self.get_classname(pkg)
+            src = self._loader.schemaComponentSource(pkg, file)
+            if not self._schema.hasComponent(src):
+                self._schema.addComponent(src)
+                self.loadComponent(src)
+
+    def loadComponent(self, src):
+        r = self._loader.openResource(src)
+        parser = ComponentParser(self._loader, src, self._schema)
+        try:
+            xml.sax.parse(r.file, parser)
+        finally:
+            r.close()
+
+    def end_import(self):
+        pass
+
+    def start_sectiontype(self, attrs):
+        name = attrs.get("name")
+        if not name:
+            self.error("sectiontype name must not be omitted or empty")
+        name = self.basic_key(name)
+        self.push_prefix(attrs)
+        if attrs.has_key("extends"):
+            basename = self.basic_key(attrs["extends"])
+            base = self._schema.gettype(basename)
+            if base.isabstract():
+                self.error("sectiontype cannot extend an abstract type")
+            keytype, valuetype, datatype = self.get_sect_typeinfo(attrs, base)
+            sectinfo = self._schema.deriveSectionType(
+                base, name, keytype, valuetype, datatype)
+        else:
+            keytype, valuetype, datatype = self.get_sect_typeinfo(attrs)
+            sectinfo = self._schema.createSectionType(
+                name, keytype, valuetype, datatype)
+        if attrs.has_key("implements"):
+            ifname = self.basic_key(attrs["implements"])
+            interface = self._schema.gettype(ifname)
+            if not interface.isabstract():
+                self.error(
+                    "type specified by implements is not an abstracttype")
+            interface.addsubtype(sectinfo)
+        self._stack.append(sectinfo)
+
+    def end_sectiontype(self):
+        self.pop_prefix()
+        self._stack.pop()
+
+    def start_section(self, attrs):
+        sectiontype = self.get_sectiontype(attrs)
+        handler = self.get_handler(attrs)
+        min = self.get_required(attrs) and 1 or 0
+        any, name, attribute = self.get_name_info(attrs, "section")
+        if any and not attribute:
+            self.error(
+                "attribute must be specified if section name is '*' or '+'")
+        section = info.SectionInfo(any or name, sectiontype,
+                                   min, 1, handler, attribute)
+        self._stack[-1].addsection(name, section)
+        self._stack.append(section)
+
+    def end_section(self):
+        self._stack.pop()
+
+    def start_multisection(self, attrs):
+        sectiontype = self.get_sectiontype(attrs)
+        min, max = self.get_ordinality(attrs)
+        any, name, attribute = self.get_name_info(attrs, "multisection")
+        if any not in ("*", "+"):
+            self.error("multisection must specify '*' or '+' for the name")
+        handler = self.get_handler(attrs)
+        section = info.SectionInfo(any or name, sectiontype,
+                                   min, max, handler, attribute)
+        self._stack[-1].addsection(name, section)
+        self._stack.append(section)
+
+    def end_multisection(self):
+        self._stack.pop()
+
+    def start_abstracttype(self, attrs):
+        name = attrs.get("name")
+        if not name:
+            self.error("abstracttype name must not be omitted or empty")
+        name = self.basic_key(name)
+        abstype = info.AbstractType(name)
+        self._schema.addtype(abstype)
+        self._stack.append(abstype)
+
+    def end_abstracttype(self):
+        self._stack.pop()
+
+    def start_key(self, attrs):
+        name, datatype, handler, attribute = self.get_key_info(attrs, "key")
+        min = self.get_required(attrs) and 1 or 0
+        key = info.KeyInfo(name, datatype, min, handler, attribute)
+        if attrs.has_key("default"):
+            if min:
+                self.error("required key cannot have a default value")
+            key.adddefault(str(attrs["default"]).strip(),
+                           self.get_position())
+        if name != "+":
+            key.finish()
+        self._stack[-1].addkey(key)
+        self._stack.append(key)
+
+    def end_key(self):
+        key = self._stack.pop()
+        if key.name == "+":
+            key.computedefault(self._stack[-1].keytype)
+            key.finish()
+
+    def start_multikey(self, attrs):
+        if attrs.has_key("default"):
+            self.error("default values for multikey must be given using"
+                       " 'default' elements")
+        name, datatype, handler, attribute = self.get_key_info(attrs,
+                                                               "multikey")
+        min, max = self.get_ordinality(attrs)
+        key = info.MultiKeyInfo(name, datatype, min, max, handler, attribute)
+        self._stack[-1].addkey(key)
+        self._stack.append(key)
+
+    def end_multikey(self):
+        multikey = self._stack.pop()
+        if multikey.name == "+":
+            multikey.computedefault(self._stack[-1].keytype)
+        multikey.finish()
+
+    # datatype conversion wrappers
+
+    def basic_key(self, s):
+        try:
+            return self._basic_key(s)
+        except ValueError, e:
+            self.error(e[0])
+
+    def identifier(self, s):
+        try:
+            return self._identifier(s)
+        except ValueError, e:
+            self.error(e[0])
+
+    # exception setup helpers
+
+    def initerror(self, e):
+        if self._locator is not None:
+            e.colno = self._locator.getColumnNumber()
+            e.lineno = self._locator.getLineNumber()
+            e.url = self._locator.getSystemId()
+        return e
+
+    def error(self, message):
+        raise self.initerror(ZConfig.SchemaError(message))
+
+
+class SchemaParser(BaseParser):
+
+    # needed by startElement() and endElement()
+    _handled_tags = BaseParser._handled_tags + ("schema",)
+    _top_level = "schema"
+
+    def __init__(self, loader, url, extending_parser=None):
+        BaseParser.__init__(self, loader, url)
+        self._extending_parser = extending_parser
+        self._base_keytypes = []
+        self._base_datatypes = []
+
+    def start_schema(self, attrs):
+        self.push_prefix(attrs)
+        handler = self.get_handler(attrs)
+        keytype, valuetype, datatype = self.get_sect_typeinfo(attrs)
+
+        if self._extending_parser is None:
+            # We're not being inherited, so we need to create the schema
+            self._schema = info.SchemaType(keytype, valuetype, datatype,
+                                           handler, self._url, self._registry)
+        else:
+            # Parse into the extending ("subclass") parser's schema
+            self._schema = self._extending_parser._schema
+
+        self._stack = [self._schema]
+
+        if attrs.has_key("extends"):
+            sources = attrs["extends"].split()
+            sources.reverse()
+
+            for src in sources:
+                src = url.urljoin(self._url, src)
+                src, fragment = url.urldefrag(src)
+                if fragment:
+                    self.error("schema extends many not include"
+                               " a fragment identifier")
+                self.extendSchema(src)
+
+            # Inherit keytype from bases, if unspecified and not conflicting
+            if self._base_keytypes and not attrs.has_key("keytype"):
+                keytype = self._base_keytypes[0]
+                for kt in self._base_keytypes[1:]:
+                    if kt is not keytype:
+                        self.error("base schemas have conflicting keytypes,"
+                                   " but no keytype was specified in the"
+                                   " extending schema")
+
+            # Inherit datatype from bases, if unspecified and not conflicting
+            if self._base_datatypes and not attrs.has_key("datatype"):
+                datatype = self._base_datatypes[0]
+                for dt in self._base_datatypes[1:]:
+                    if dt is not datatype:
+                        self.error("base schemas have conflicting datatypes,"
+                                   " but no datatype was specified in the"
+                                   " extending schema")
+
+        # Reset the schema types to our own, while we parse the schema body
+        self._schema.keytype = keytype
+        self._schema.valuetype = valuetype
+        self._schema.datatype = datatype
+
+        # Update base key/datatypes for the "extending" parser
+        if self._extending_parser is not None:
+            self._extending_parser._base_keytypes.append(keytype)
+            self._extending_parser._base_datatypes.append(datatype)
+
+
+    def extendSchema(self,src):
+        parser = SchemaParser(self._loader, src, self)
+        r = self._loader.openResource(src)
+        try:
+            xml.sax.parse(r.file, parser)
+        finally:
+            r.close()
+
+    def end_schema(self):
+        del self._stack[-1]
+        assert not self._stack
+        self.pop_prefix()
+        assert not self._prefixes
+
+
+class ComponentParser(BaseParser):
+
+    _handled_tags = BaseParser._handled_tags + ("component",)
+    _top_level = "component"
+
+    def __init__(self, loader, url, schema):
+        BaseParser.__init__(self, loader, url)
+        self._parent = schema
+
+    def characters_description(self, data):
+        if self._stack:
+            self._stack[-1].description = data
+
+    def start_key(self, attrs):
+        self._check_not_toplevel("key")
+        BaseParser.start_key(self, attrs)
+
+    def start_multikey(self, attrs):
+        self._check_not_toplevel("multikey")
+        BaseParser.start_multikey(self, attrs)
+
+    def start_section(self, attrs):
+        self._check_not_toplevel("section")
+        BaseParser.start_section(self, attrs)
+
+    def start_multisection(self, attrs):
+        self._check_not_toplevel("multisection")
+        BaseParser.start_multisection(self, attrs)
+
+    def start_component(self, attrs):
+        self._schema = self._parent
+        self.push_prefix(attrs)
+
+    def end_component(self):
+        self.pop_prefix()
+
+    def _check_not_toplevel(self, what):
+        if not self._stack:
+            self.error("cannot define top-level %s in a schema %s"
+                       % (what, self._top_level))

trunk/src/ZConfig/scripts/zconfig

+#!/usr/bin/env python
+##############################################################################
+#
+# Copyright (c) 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+"""zconfig: Script to check validity of a configuration file.
+
+Usage:
+
+    zconfig [options] [file...]
+
+Options:
+
+    -h
+    --help          Print this help text.
+
+    -s file
+    --schema file   Use the schema in 'file' to validate the configuration;
+                    this must be specified.
+
+Each file named on the command line is checked for syntactical errors
+and schema conformance.  The schema must be specified.  If no files
+are specified and standard input is not a TTY, standard in is treated
+as a configuration file.  Specifying a schema and no configuration
+files causes the schema to be checked.
+
+"""
+
+import optparse
+import sys
+
+import ZConfig
+
+
+def main():
+    optparser = optparse.OptionParser(
+        usage="usage: %prog [-s FILE] [file ...]")
+    optparser.add_option(
+        "-s", "--schema", dest="schema",
+        help="use the schema in FILE (can be a URL)",
+        metavar="FILE")
+    options, args = optparser.parse_args()
+
+    if not options.schema:
+        print >>sys.stderr, "No schema specified, but is required."
+        usage(sys.stderr)
+        return 2
+    schema = ZConfig.loadSchema(options.schema)
+
+    if not args:
+        if sys.stdin.isatty():
+            # just checking the schema
+            return 0
+        else:
+            # stdin is a pipe
+            args = ["-"]
+
+    errors = 0
+    for fn in args:
+        try:
+            if fn == "-":
+                ZConfig.loadConfigFile(schema, sys.stdin)
+            else:
+                ZConfig.loadConfig(schema, fn)
+        except ZConfig.ConfigurationError, e:
+            print >>sys.stderr, str(e)
+            errors += 1
+
+    if errors:
+        return 1
+    else:
+        return 0
+
+
+def usage(fp):
+    print >>fp, __doc__
+
+
+if __name__ == "__main__":
+    sys.exit(main())

trunk/src/ZConfig/scripts/zconfig_schema2html

+#!/usr/bin/env python
+##############################################################################
+#
+# Copyright (c) 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+__version__ = '$Revision: 1.3 $'[11:-2]
+
+import ZConfig.loader
+from ZConfig.info import *
+import sys, cgi
+
+def esc(x): return cgi.escape(str(x))
+def dt(x):
+    tn = type(x).__name__
+    if tn == 'instance':
+        return '%s %s'%(tn, x.__class__.__module__ + '.' + x.__class__.__name__)
+    elif tn == 'class':
+        return '%s %s'%(tn, x.__module__ + '.' + x.__name__)
+    else:
+        return '%s %s'%(tn, x.__name__)
+
+class explain:
+    done = []
+    def __call__(self, st):
+        if st.name in self.done:
+            return
+        self.done.append(st.name)
+
+        if st.description:
+            print st.description
+        for sub in st.getsubtypenames():
+            print '<dl>'
+            printContents(None, st.getsubtype(sub))
+            print '</dl>'
+explain = explain()
+
+def printSchema(schema):
+    print '<dl>'
+    for child in schema:
+        printContents(*child)
+    print '</dl>'
+
+def printContents(name, info):
+    if isinstance(info, SectionType):
+        print '<dt><b><i>', info.name, '</i></b> (%s)</dt>'%dt(info.datatype)
+        print '<dd>'
+        if info.description:
+            print info.description
+        print '<dl>'
+        for sub in info:
+            printContents(*sub)
+        print '</dl></dd>'
+    elif isinstance(info, SectionInfo):
+        st = info.sectiontype
+        if st.isabstract():
+            print '<dt><b><i>', st.name, '</i>', info.name, '</b></dt>'
+            print '<dd>'
+            if info.description:
+                print info.description
+            explain(st)
+            print '</dd>'
+        else:
+            print '<dt><b>', info.attribute, info.name, '</b>'
+            print '(%s)</dt>'%dt(info.datatype)
+            print '<dd><dl>'
+            for sub in info.sectiontype:
+                printContents(*sub)
+            print '</dl></dd>'
+    else:
+        print '<dt><b>',info.name, '</b>', '(%s)'%dt(info.datatype)
+        default = info.getdefault()
+        if isinstance(default, ValueInfo):
+            print '(default: %r)'%esc(default.value)
+        elif default is not None:
+            print '(default: %r)'%esc(default)
+        if info.metadefault:
+            print '(metadefault: %s)' % info.metadefault
+        print '</dt>'
+        if info.description:
+            print '<dd>',info.description,'</dd>'
+
+schema = ZConfig.loader.loadSchemaFile(sys.argv[1])
+
+print '''<html><body>
+<style>
+dl {margin: 0 0 1em 0;}
+</style>
+'''
+printSchema(schema)
+print '</body></html>'
+
+# vim: set filetype=python ts=4 sw=4 et si
+

trunk/src/ZConfig/substitution.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Substitution support for ZConfig values."""
+
+import ZConfig
+
+
+def substitute(s, mapping):
+    """Interpolate variables from `mapping` into `s`."""
+    if "$" in s:
+        result = ''
+        rest = s
+        while rest:
+            p, name, namecase, rest = _split(rest)
+            result += p
+            if name:
+                v = mapping.get(name)
+                if v is None:
+                    raise ZConfig.SubstitutionReplacementError(s, namecase)
+                result += v
+        return result
+    else:
+        return s
+
+
+def isname(s):
+    """Return True iff s is a valid substitution name."""
+    m = _name_match(s)
+    if m:
+        return m.group() == s
+    else:
+        return False
+
+
+def _split(s):
+    # Return a four tuple:  prefix, name, namecase, suffix
+    # - prefix is text that can be used literally in the result (may be '')
+    # - name is a referenced name, or None
+    # - namecase is the name with case preserved
+    # - suffix is trailling text that may contain additional references
+    #   (may be '' or None)
+    if "$" in s:
+        i = s.find("$")
+        c = s[i+1:i+2]
+        if c == "":
+            raise ZConfig.SubstitutionSyntaxError(
+                "illegal lone '$' at end of source")
+        if c == "$":
+            return s[:i+1], None, None, s[i+2:]
+        prefix = s[:i]
+        if c == "{":
+            m = _name_match(s, i + 2)
+            if not m:
+                raise ZConfig.SubstitutionSyntaxError(
+                    "'${' not followed by name")
+            name = m.group(0)
+            i = m.end() + 1
+            if not s.startswith("}", i - 1):
+                raise ZConfig.SubstitutionSyntaxError(
+                    "'${%s' not followed by '}'" % name)
+        else:
+            m = _name_match(s, i+1)
+            if not m:
+                raise ZConfig.SubstitutionSyntaxError(
+                    "'$' not followed by '$' or name")
+            name = m.group(0)
+            i = m.end()
+        return prefix, name.lower(), name, s[i:]
+    else:
+        return s, None, None, None
+
+
+import re
+_name_match = re.compile(r"[a-zA-Z_][a-zA-Z0-9_]*").match
+del re

trunk/src/ZConfig/tests/__init__.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Tests for the configuration data structures and loader.
+
+$Id: __init__.py,v 1.2 2003/01/03 21:05:56 fdrake Exp $
+"""

trunk/src/ZConfig/tests/input/base-datatype1.xml

+<schema datatype="ZConfig.tests.test_schema.MySection">
+  <sectiontype name="type-1"/>
+</schema>
+

trunk/src/ZConfig/tests/input/base-datatype2.xml

+<schema datatype="ZConfig.tests.test_schema.appsection">
+  <sectiontype name="type-2"/>
+</schema>

trunk/src/ZConfig/tests/input/base-keytype1.xml

+<schema keytype="basic-key">
+  <sectiontype name="type-1"/>
+</schema>

trunk/src/ZConfig/tests/input/base-keytype2.xml

+<schema keytype="ZConfig.tests.test_schema.uppercase">
+  <sectiontype name="type-2"/>
+</schema>

trunk/src/ZConfig/tests/input/base.xml

+<schema>
+  <sectiontype name="type-X"/>
+</schema>

trunk/src/ZConfig/tests/input/include.conf

+var2 value2
+%include simple.conf
+var3 value3
+var4 $name

trunk/src/ZConfig/tests/input/inner.conf

+refouter $outervar
+%define innervar inner

trunk/src/ZConfig/tests/input/library.xml

+<schema>
+  <description>
+    Sample library of reusable data types.
+  </description>
+  <sectiontype name="type-a"/>
+  <sectiontype name="type-b"/>
+</schema>

trunk/src/ZConfig/tests/input/logger.xml

+<schema>
+  <sectiontype name="loghandler">
+    <key name="constructor" datatype="constructor" required="yes"/>
+    <key name="formatter" datatype="constructor"
+         default="logging.Formatter()"/>
+  </sectiontype>
+
+  <sectiontype name="logger">
+     <key name="level" datatype="integer" default="info"/>
+     <multisection type="loghandler" attribute="handlers" name="*"/>
+  </sectiontype>
+</schema>

trunk/src/ZConfig/tests/input/outer.conf

+%define outervar outer
+%include inner.conf
+refinner $innervar

trunk/src/ZConfig/tests/input/simple.conf

+empty
+
+var1 abc
+int-var  12
+float-var	12.02
+neg-int -2
+
+true-var-1 true
+true-var-2 on
+true-var-3 yes
+
+false-var-1 false
+false-var-2 off
+false-var-3 no
+
+list-1
+list-2 abc
+list-3 abc def ghi
+list-4 	[  what	now? 	]
+
+# These test the %define mechanism:
+
+%define dollars $$$$
+%define empty
+%define name value
+%define twowords two words
+
+getname $name
+getnametwice $name${name}
+getdollars $dollars
+getempty x${empty}y
+getwords abc $twowords def

trunk/src/ZConfig/tests/input/simple.xml

+<schema>
+  <key name="empty"        />
+  <key name="var1"         />
+  <key name="getname"      />
+  <key name="getnametwice" />
+  <key name="getdollars"   />
+  <key name="getempty"     />
+  <key name="getwords"     />
+
+  <key name="int-var"     datatype="integer" />
+  <key name="float-var"   datatype="float"   />
+  <key name="neg-int"     datatype="integer" />
+  <key name="true-var-1"  datatype="boolean" />
+  <key name="true-var-2"  datatype="boolean" />
+  <key name="true-var-3"  datatype="boolean" />
+  <key name="false-var-1" datatype="boolean" />
+  <key name="false-var-2" datatype="boolean" />
+  <key name="false-var-3" datatype="boolean" />
+
+  <key name="list-1" datatype="string-list" />
+  <key name="list-2" datatype="string-list" />
+  <key name="list-3" datatype="string-list" />
+  <key name="list-4" datatype="string-list" />
+
+  <!-- added by include.conf -->
+  <key name="var2" />
+  <key name="var3" />
+  <key name="var4" />
+</schema>

trunk/src/ZConfig/tests/input/simplesections.conf

+var foo
+var-0 foo-0
+
+<section name>
+  var bar
+  var-one  splat
+</section>
+
+var-1 foo-1
+
+<section delegate>
+  var spam
+  var-two  stuff
+</section>
+
+var-2 foo-2
+
+<section another>
+  var  quack!
+  var-three  yet
+</section>
+
+var-3 foo-3
+
+# An anonymous empty section:
+<section />
+
+var-4 foo-4
+
+# A fairly trivial section:
+<trivial>
+  var triv
+</trivial>
+
+var-5 foo-5
+
+# A minimal section:
+<minimal/>
+
+var-6 foo-6

trunk/src/ZConfig/tests/input/simplesections.xml

+<schema>
+  <sectiontype name="section">
+    <key name="var" />
+    <key name="var-one" />
+    <key name="var-two" />
+    <key name="var-three" />
+  </sectiontype>
+  <sectiontype name="minimal" />
+  <sectiontype name="trivial">
+    <key name="var" />
+  </sectiontype>
+
+  <multisection type="section" name="*" attribute="sections" />
+  <section type="minimal" name="*" attribute="minimal" />
+  <section type="trivial" name="*" attribute="trivial" />
+
+  <key name="var" />
+  <key name="var-0" />
+  <key name="var-1" />
+  <key name="var-2" />
+  <key name="var-3" />
+  <key name="var-4" />
+  <key name="var-5" />
+  <key name="var-6" />
+</schema>

trunk/src/ZConfig/tests/library/README.txt

+This is a sample library of configuration schema components.  This is
+used for testing.

trunk/src/ZConfig/tests/library/__init__.py

+# Make this a package.

trunk/src/ZConfig/tests/library/thing/__init__.py

+##############################################################################
+#
+# Copyright (c) 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Example of a package that extends its __path__.
+
+$Id: __init__.py,v 1.2 2003/10/03 17:11:33 fdrake Exp $
+"""
+
+import os
+
+here = os.path.dirname(__file__)
+__path__.append(os.path.join(here, "extras"))

trunk/src/ZConfig/tests/library/thing/component.xml

+<?xml version="1.0" encoding="us-ascii"?>
+<component>
+  <abstracttype name='thing'/>
+  <sectiontype name='thing-a' implements='thing'>
+    <key name='thing-a-key' default='thing-a-default'/>
+  </sectiontype>
+  <sectiontype name='thing-b' implements='thing'>
+    <key name='thing-b-key' default='thing-b-default'/>
+  </sectiontype>
+</component>

trunk/src/ZConfig/tests/library/thing/extras/extras.xml

+<component>
+  <sectiontype name='extra-thing'>
+    <key name='some-key'/>
+  </sectiontype>
+</component>

trunk/src/ZConfig/tests/library/widget/__init__.py

+# Make this a package.

trunk/src/ZConfig/tests/library/widget/component.xml

+<component>
+  <abstracttype name='widget'/>
+  <sectiontype name='widget-a' implements='widget'>
+    <key name='widget-a-key' default='widget-a-default'/>
+  </sectiontype>
+  <sectiontype name='widget-b' implements='widget' extends='widget-a'/>
+</component>

trunk/src/ZConfig/tests/library/widget/extra.xml

+<component>
+  <sectiontype name='extra-type'>
+    <key name='some-key'/>
+  </sectiontype>
+</component>

trunk/src/ZConfig/tests/runtests.bat

+@echo off
+
+rem  Simple script to run the tests on Windows.
+
+rem  The paths to different versions of Python need to be
+rem  edited for the system this is being run on; comment
+rem  out lines that aren't needed or wanted.
+
+\Python213\python runtests.py
+\Python221\python runtests.py
+\Python222\python runtests.py
+\Python230\python runtests.py

trunk/src/ZConfig/tests/runtests.py

+#! /usr/bin/env python
+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Script to run all the regression tests for the ZConfig package."""
+
+import os
+import sys
+import unittest
+
+if __name__ == "__main__":
+    __file__ = sys.argv[0]
+
+TESTDIR = os.path.dirname(os.path.abspath(__file__))
+
+PKGDIR = os.path.dirname(TESTDIR) # the ZConfig package directory
+TOPDIR = os.path.dirname(PKGDIR)
+
+COMPONENTS = os.path.join(PKGDIR, "components")
+
+TESTDIRS = {
+    "ZConfig.tests": TESTDIR,
+    "ZConfig.components.basic.tests": os.path.join(COMPONENTS,
+                                                   "basic", "tests"),
+    "ZConfig.components.logger.tests": os.path.join(COMPONENTS,
+                                                    "logger", "tests"),
+    }
+
+
+def load_tests(pkgname, name):
+    name = "%s.%s" % (pkgname, name)
+    __import__(name)
+    mod = sys.modules[name]
+    return mod.test_suite()
+
+def test_suite():
+    L = []
+    for pkgname, path in TESTDIRS.items():
+        for fn in os.listdir(path):
+            name, ext = os.path.splitext(fn)
+            if name[:4] == "test" and ext == ".py":
+                L.append(load_tests(pkgname, name))
+    if len(L) == 1:
+        return L[0]
+    else:
+        suite = unittest.TestSuite()
+        for t in L:
+            suite.addTest(t)
+        return suite
+
+if __name__ == "__main__":
+    if TOPDIR not in sys.path:
+        sys.path.insert(0, TOPDIR)
+    unittest.main(defaultTest="test_suite")

trunk/src/ZConfig/tests/support.py

+##############################################################################
+#
+# Copyright (c) 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+"""Support code shared among the tests."""
+
+import os
+import StringIO
+import unittest
+import urllib
+
+import ZConfig
+
+from ZConfig.loader import ConfigLoader
+from ZConfig.url import urljoin
+
+
+try:
+    __file__
+except NameError:
+    import sys
+    __file__ = sys.argv[0]
+
+d = os.path.abspath(os.path.join(os.path.dirname(__file__), "input"))
+CONFIG_BASE = "file://%s/" % urllib.pathname2url(d)
+
+
+class TestBase(unittest.TestCase):
+    """Utility methods which can be used with the schema support."""
+
+    def load_both(self, schema_url, conf_url):
+        schema = self.load_schema(schema_url)
+        conf = self.load_config(schema, conf_url)
+        return schema, conf
+
+    def load_schema(self, relurl):
+        self.url = urljoin(CONFIG_BASE, relurl)
+        self.schema = ZConfig.loadSchema(self.url)
+        self.assert_(self.schema.issection())
+        return self.schema
+
+    def load_schema_text(self, text, url=None):
+        sio = StringIO.StringIO(text)
+        self.schema = ZConfig.loadSchemaFile(sio, url)
+        return self.schema
+
+    def load_config(self, schema, conf_url, num_handlers=0):
+        conf_url = urljoin(CONFIG_BASE, conf_url)
+        loader = self.create_config_loader(schema)
+        self.conf, self.handlers = loader.loadURL(conf_url)
+        self.assertEqual(len(self.handlers), num_handlers)
+        return self.conf
+
+    def load_config_text(self, schema, text, num_handlers=0, url=None):
+        sio = StringIO.StringIO(text)
+        loader = self.create_config_loader(schema)
+        self.conf, self.handlers = loader.loadFile(sio, url)
+        self.assertEqual(len(self.handlers), num_handlers)
+        return self.conf
+
+    def create_config_loader(self, schema):
+        return ConfigLoader(schema)

trunk/src/ZConfig/tests/test_cfgimports.py

+##############################################################################
+#
+# Copyright (c) 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Tests of the %import mechanism.
+
+$Id: test_cfgimports.py,v 1.1 2003/10/03 20:01:57 fdrake Exp $
+"""
+
+import unittest
+
+from StringIO import StringIO
+
+import ZConfig
+import ZConfig.tests.support
+
+
+class TestImportFromConfiguration(ZConfig.tests.support.TestBase):
+
+    def test_simple_import(self):
+        schema = self.load_schema_text("<schema/>")
+        loader = self.create_config_loader(schema)
+        config, _ = loader.loadFile(
+            StringIO("%import ZConfig.tests.library.widget\n"))
+        # make sure we now have a "private" schema object; the only
+        # way to get it is from the loader itself
+        self.assert_(schema is not loader.schema)
+        # make sure component types are only found on the private schema:
+        loader.schema.gettype("widget-b")
+        self.assertRaises(ZConfig.SchemaError, schema.gettype, "widget-b")
+
+    def test_repeated_import(self):
+        schema = self.load_schema_text("<schema/>")
+        loader = self.create_config_loader(schema)
+        config, _ = loader.loadFile(
+            StringIO("%import ZConfig.tests.library.widget\n"
+                     "%import ZConfig.tests.library.widget\n"))
+
+    def test_missing_import(self):
+        schema = self.load_schema_text("<schema/>")
+        loader = self.create_config_loader(schema)
+        self.assertRaises(ZConfig.SchemaError, loader.loadFile,
+                          StringIO("%import ZConfig.tests.missing\n"))
+
+
+def test_suite():
+    return unittest.makeSuite(TestImportFromConfiguration)

trunk/src/ZConfig/tests/test_cmdline.py

+##############################################################################
+#
+# Copyright (c) 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+"""Tests of the command-line integration."""
+
+import unittest
+
+import ZConfig
+
+from ZConfig.cmdline import ExtendedConfigLoader
+from ZConfig.tests.support import TestBase
+
+
+class CommandLineTest(TestBase):
+
+    def create_config_loader(self, schema):
+        loader = ExtendedConfigLoader(schema)
+        for item in self.clopts:
+            loader.addOption(*item)
+        return loader
+
+    def test_loading(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='st'>
+                <key name='innerkey'/>
+              </sectiontype>
+              <key name='mykey'/>
+              <section name='*' type='st' attribute='sect'/>
+            </schema>
+            """)
+        self.clopts = [("mykey=splat!", None),
+                       ("section/innerkey=spoogey", None)]
+        bag = self.create_config_loader(schema).cook()
+        # Test a variety of queries on the OptionBag:
+        self.assert_(bag.has_key("mykey"))
+        self.assert_(not bag.has_key("another"))
+        self.assertEqual(bag.get_section_info("st", None), None)
+        self.assertEqual(bag.get_section_info("st", "missing-sect"), None)
+        # Consume everything in the OptionBag:
+        L = bag.get_key("mykey")
+        s, pos = L[0]
+        self.assertEqual(len(L), 1)
+        self.assertEqual(s, "splat!")
+        bag2 = bag.get_section_info("st", "section")
+        self.assert_(bag2.has_key("innerkey"))
+        self.assert_(not bag2.has_key("another"))
+        L = bag2.get_key("innerkey")
+        s, pos = L[0]
+        self.assertEqual(len(L), 1)
+        self.assertEqual(s, "spoogey")
+        # "Finish" to make sure everything has been consumed:
+        bag2.finish()
+        bag.finish()
+
+    def test_named_sections(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <abstracttype name='at'/>
+              <sectiontype name='st1' implements='at'>
+                <key name='k1'/>
+              </sectiontype>
+              <sectiontype name='st2' implements='at'>
+                <key name='k2'/>
+              </sectiontype>
+              <section name='foo' type='at'/>
+              <section name='bar' type='st2'/>
+            </schema>
+            """)
+        self.clopts = [("foo/k1=v1", None), ("bar/k2=v2", ("someurl", 2, 3))]
+        bag = self.create_config_loader(schema).cook()
+        foo = bag.get_section_info("st2", "foo")
+        bar = bag.get_section_info("st2", "bar")
+        bag.finish()
+        self.assertEqual(bar.get_key("k2"), [("v2", ("someurl", 2, 3))])
+        bar.finish()
+        # Ignore foo for now; it's not really important *when* it fails.
+
+    simple_schema = None
+
+    def get_simple_schema(self):
+        if self.simple_schema is None:
+            self.__class__.simple_schema = self.load_schema_text("""\
+                <schema>
+                  <key name='k0'/>
+                  <key name='k1'/>
+                  <key name='k2' datatype='integer'/>
+                  <key name='k3' datatype='integer' default='19'/>
+                </schema>
+                """)
+        return self.simple_schema
+
+    def test_reading_config(self):
+        self.clopts = [("k1=stringvalue", None), ("k2=12", None)]
+        schema = self.get_simple_schema()
+        conf = self.load_config_text(schema, """\
+            k0 stuff
+            k1 replaced-stuff
+            k2 42
+            """)
+        self.assertEqual(conf.k0, "stuff")
+        self.assertEqual(conf.k1, "stringvalue")
+        self.assertEqual(conf.k2, 12)
+        self.assertEqual(conf.k3, 19)
+
+    def test_unknown_key(self):
+        self.clopts = [("foo=bar", None)]
+        schema = self.get_simple_schema()
+        self.assertRaises(ZConfig.ConfigurationError,
+                          self.load_config_text, schema, "")
+
+    def test_too_many_keys(self):
+        self.clopts = [("k1=v1", None), ("k1=v2", None)]
+        schema = self.get_simple_schema()
+        self.assertRaises(ZConfig.ConfigurationError,
+                          self.load_config_text, schema, "")
+
+    def test_bad_datatype(self):
+        self.clopts = [("k2=42.0", None)]
+        schema = self.get_simple_schema()
+        self.assertRaises(ZConfig.DataConversionError,
+                          self.load_config_text, schema, "")
+
+    def test_without_clopts(self):
+        self.clopts = []
+        schema = self.get_simple_schema()
+        conf = self.load_config_text(schema, "k3 42")
+        self.assertEqual(conf.k0, None)
+        self.assertEqual(conf.k1, None)
+        self.assertEqual(conf.k2, None)
+        self.assertEqual(conf.k3, 42)
+
+    def test_section_contents(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='st'>
+                <key name='k1'/>
+                <key name='k2' default='3' datatype='integer'/>
+                <multikey name='k3'>
+                  <default>k3-v1</default>
+                  <default>k3-v2</default>
+                  <default>k3-v3</default>
+                </multikey>
+              </sectiontype>
+              <section name='s1' type='st'/>
+              <section name='s2' type='st'/>
+            </schema>
+            """)
+        self.clopts = [("s1/k1=foo", None),
+                       ("s2/k3=value1", None),
+                       ("s2/k3=value2", None),
+                       ("s1/k2=99", None),
+                       ("s2/k3=value3", None),
+                       ("s2/k3=value4", None),
+                       ]
+        conf = self.load_config_text(schema, "<st s1/>\n<st s2/>")
+        self.assertEqual(conf.s1.k1, "foo")
+        self.assertEqual(conf.s1.k2, 99)
+        self.assertEqual(conf.s1.k3, ["k3-v1", "k3-v2", "k3-v3"])
+        self.assertEqual(conf.s2.k1, None)
+        self.assertEqual(conf.s2.k2, 3)
+        self.assertEqual(conf.s2.k3, ["value1", "value2", "value3", "value4"])
+
+
+def test_suite():
+    return unittest.makeSuite(CommandLineTest)
+
+if __name__ == "__main__":
+    unittest.main(defaultTest="test_suite")

trunk/src/ZConfig/tests/test_config.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Tests of the configuration data structures and loader."""
+
+import os
+import StringIO
+import tempfile
+import unittest
+
+import ZConfig
+
+from ZConfig.tests.support import CONFIG_BASE
+
+
+class ConfigurationTestCase(unittest.TestCase):
+
+    schema = None
+
+    def get_schema(self):
+        if self.schema is None:
+            ConfigurationTestCase.schema = ZConfig.loadSchema(
+                CONFIG_BASE + "simple.xml")
+        return self.schema
+
+    def load(self, relurl, context=None):
+        url = CONFIG_BASE + relurl
+        self.conf, self.handlers = ZConfig.loadConfig(self.get_schema(), url)
+        conf = self.conf
+        #self.assertEqual(conf.url, url)
+        self.assert_(conf.getSectionName() is None)
+        self.assert_(conf.getSectionType() is None)
+        #self.assert_(conf.delegate is None)
+        return conf
+
+    def loadtext(self, text):
+        sio = StringIO.StringIO(text)
+        return self.loadfile(sio)
+
+    def loadfile(self, file):
+        schema = self.get_schema()
+        self.conf, self.handlers = ZConfig.loadConfigFile(schema, file)
+        return self.conf
+
+    def check_simple_gets(self, conf):
+        self.assertEqual(conf.empty, '')
+        self.assertEqual(conf.int_var, 12)
+        self.assertEqual(conf.neg_int, -2)
+        self.assertEqual(conf.float_var, 12.02)
+        self.assertEqual(conf.var1, 'abc')
+        self.assert_(conf.true_var_1)
+        self.assert_(conf.true_var_2)
+        self.assert_(conf.true_var_3)
+        self.assert_(not conf.false_var_1)
+        self.assert_(not conf.false_var_2)
+        self.assert_(not conf.false_var_3)
+        self.assertEqual(conf.list_1, [])
+        self.assertEqual(conf.list_2, ['abc'])
+        self.assertEqual(conf.list_3, ['abc', 'def', 'ghi'])
+        self.assertEqual(conf.list_4, ['[', 'what', 'now?', ']'])
+
+    def test_simple_gets(self):
+        conf = self.load("simple.conf")
+        self.check_simple_gets(conf)
+
+    def test_type_errors(self):
+        Error = ZConfig.DataConversionError
+        raises = self.assertRaises
+        raises(Error, self.loadtext, "int-var true")
+        raises(Error, self.loadtext, "float-var true")
+        raises(Error, self.loadtext, "neg-int false")
+        raises(Error, self.loadtext, "true-var-1 0")
+        raises(Error, self.loadtext, "true-var-1 1")
+        raises(Error, self.loadtext, "true-var-1 -1")
+
+    def test_simple_sections(self):
+        self.schema = ZConfig.loadSchema(CONFIG_BASE + "simplesections.xml")
+        conf = self.load("simplesections.conf")
+        self.assertEqual(conf.var, "foo")
+        # check each interleaved position between sections
+        for c in "0123456":
+            self.assertEqual(getattr(conf, "var_" +c), "foo-" + c)
+        sect = [sect for sect in conf.sections
+                if sect.getSectionName() == "name"][0]
+        self.assertEqual(sect.var, "bar")
+        self.assertEqual(sect.var_one, "splat")
+        self.assert_(sect.var_three is None)
+        sect = [sect for sect in conf.sections
+                if sect.getSectionName() == "delegate"][0]
+        self.assertEqual(sect.var, "spam")
+        self.assertEqual(sect.var_two, "stuff")
+        self.assert_(sect.var_three is None)
+
+    def test_include(self):
+        conf = self.load("include.conf")
+        self.assertEqual(conf.var1, "abc")
+        self.assertEqual(conf.var2, "value2")
+        self.assertEqual(conf.var3, "value3")
+        self.assertEqual(conf.var4, "value")
+
+    def test_includes_with_defines(self):
+        self.schema = ZConfig.loadSchemaFile(StringIO.StringIO("""\
+            <schema>
+              <key name='refinner' />
+              <key name='refouter' />
+            </schema>
+            """))
+        conf = self.load("outer.conf")
+        self.assertEqual(conf.refinner, "inner")
+        self.assertEqual(conf.refouter, "outer")
+
+    def test_define(self):
+        conf = self.load("simple.conf")
+        self.assertEqual(conf.getname, "value")
+        self.assertEqual(conf.getnametwice, "valuevalue")
+        self.assertEqual(conf.getdollars, "$$")
+        self.assertEqual(conf.getempty, "xy")
+        self.assertEqual(conf.getwords, "abc two words def")
+
+    def test_define_errors(self):
+        self.assertRaises(ZConfig.ConfigurationSyntaxError,
+                          self.loadtext, "%define\n")
+        self.assertRaises(ZConfig.ConfigurationSyntaxError,
+                          self.loadtext, "%define abc-def\n")
+        self.assertRaises(ZConfig.ConfigurationSyntaxError,
+                          self.loadtext, "%define a value\n%define a value\n")
+
+    def test_fragment_ident_disallowed(self):
+        self.assertRaises(ZConfig.ConfigurationError,
+                          self.load, "simplesections.conf#another")
+
+    def test_load_from_fileobj(self):
+        sio = StringIO.StringIO("%define name value\n"
+                                "getname x $name y \n")
+        cf = self.loadfile(sio)
+        self.assertEqual(cf.getname, "x value y")
+
+    def test_load_from_abspath(self):
+        fn = self.write_tempfile()
+        try:
+            self.check_load_from_path(fn)
+        finally:
+            os.unlink(fn)
+
+    def test_load_from_relpath(self):
+        fn = self.write_tempfile()
+        dir, name = os.path.split(fn)
+        pwd = os.getcwd()
+        try:
+            os.chdir(dir)
+            self.check_load_from_path(name)
+        finally:
+            os.chdir(pwd)
+            os.unlink(fn)
+
+    def write_tempfile(self):
+        fn = tempfile.mktemp()
+        fp = open(fn, "w")
+        fp.write("var1 value\n")
+        fp.close()
+        return fn
+
+    def check_load_from_path(self, path):
+        schema = self.get_schema()
+        ZConfig.loadConfig(schema, path)
+
+
+def test_suite():
+    return unittest.makeSuite(ConfigurationTestCase)
+
+if __name__ == '__main__':
+    unittest.main(defaultTest='test_suite')

trunk/src/ZConfig/tests/test_cookbook.py

+##############################################################################
+#
+# Copyright (c) 2004 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Tests of examples from the online cookbook, so we don't break them
+down the road.  Unless we really mean to.
+
+The ZConfig Cookbook is available online at:
+
+    http://dev.zope.org/Zope3/ZConfig
+
+"""
+
+import unittest
+
+from ZConfig.tests.support import TestBase
+
+
+def basic_key_mapping_password_to_passwd(key):
+    # Lower-case the key since that's what basic-key does:
+    key = key.lower()
+    # Now map password to passwd:
+    if key == "password":
+        key = "passwd"
+    return key
+
+def user_info_conversion(section):
+    return section
+
+
+class CookbookTestCase(TestBase):
+
+    def test_rewriting_key_names(self):
+        schema = self.load_schema_text("""
+            <schema prefix='%s'>
+              <sectiontype name='userinfo' datatype='.user_info_conversion'
+                           keytype='.basic_key_mapping_password_to_passwd'>
+                <key name='userid' datatype='integer'/>
+                <key name='username' datatype='identifier'/>
+                <key name='password'/>
+              </sectiontype>
+              <section type='userinfo' name='*' attribute='userinfo'/>
+            </schema>
+            """ % __name__)
+        config = self.load_config_text(schema, """\
+            <userinfo>
+              USERID 42
+              USERNAME foouser
+              PASSWORD yeah-right
+            </userinfo>
+            """)
+        self.assertEqual(config.userinfo.userid, 42)
+        self.assertEqual(config.userinfo.username, "foouser")
+        self.assertEqual(config.userinfo.passwd, "yeah-right")
+        self.assert_(not hasattr(config.userinfo, "password"))
+
+
+def test_suite():
+    return unittest.makeSuite(CookbookTestCase)
+
+if __name__ == "__main__":
+    unittest.main(defaultTest="test_suite")

trunk/src/ZConfig/tests/test_datatypes.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Tests of standard ZConfig datatypes."""
+
+import os
+import sys
+import shutil
+import socket
+import datetime
+import tempfile
+import unittest
+
+import ZConfig.datatypes
+
+try:
+    here = __file__
+except NameError:
+    here = sys.argv[0]
+
+here = os.path.abspath(here)
+
+try:
+    unicode
+except NameError:
+    have_unicode = False
+else:
+    have_unicode = True
+
+
+class DatatypeTestCase(unittest.TestCase):
+    types = ZConfig.datatypes.Registry()
+
+    def test_datatype_basickey(self):
+        convert = self.types.get("basic-key")
+        eq = self.assertEqual
+        raises = self.assertRaises
+
+        eq(convert("abc"), "abc")
+        eq(convert("ABC_DEF.123"), "abc_def.123")
+        eq(convert("Abc-Def-456"), "abc-def-456")
+        eq(convert("Abc.Def"), "abc.def")
+
+        raises(ValueError, convert, "_abc")
+        raises(ValueError, convert, "-abc")
+        raises(ValueError, convert, "123")
+        raises(ValueError, convert, "")
+
+    def test_datatype_boolean(self):
+        convert = self.types.get("boolean")
+        check = self.assert_
+        raises = self.assertRaises
+
+        check(convert("on"))
+        check(convert("true"))
+        check(convert("yes"))
+        check(not convert("off"))
+        check(not convert("false"))
+        check(not convert("no"))
+        raises(ValueError, convert, '0')
+        raises(ValueError, convert, '1')
+        raises(ValueError, convert, '')
+        raises(ValueError, convert, 'junk')
+
+    def test_datatype_float(self):
+        convert = self.types.get("float")
+        eq = self.assertEqual
+        raises = self.assertRaises
+
+        eq(convert("1"), 1.0)
+        self.assert_(type(convert(1)) is type(1.0))
+        eq(convert("1.1"), 1.1)
+        eq(convert("50.50"), 50.50)
+        eq(convert("-50.50"), -50.50)
+        eq(convert(0), 0.0)
+        eq(convert("0"), 0.0)
+        eq(convert("-0"), 0.0)
+        eq(convert("0.0"), 0.0)
+
+        raises(ValueError, convert, "junk")
+        raises(ValueError, convert, "0x234.1.9")
+        raises(ValueError, convert, "0.9-")
+
+        # These are not portable representations; make sure they are
+        # disallowed everywhere for consistency.
+        raises(ValueError, convert, "inf")
+        raises(ValueError, convert, "-inf")
+        raises(ValueError, convert, "nan")
+
+        if have_unicode:
+            raises(ValueError, convert, unicode("inf"))
+            raises(ValueError, convert, unicode("-inf"))
+            raises(ValueError, convert, unicode("nan"))
+
+    def test_datatype_identifier(self):
+        convert = self.types.get("identifier")
+        raises = self.assertRaises
+        self.check_names(convert)
+        self.check_never_namelike(convert)
+        raises(ValueError, convert, ".abc")
+
+    def check_names(self, convert):
+        eq = self.assert_ascii_equal
+        eq(convert, "AbcDef")
+        eq(convert, "a________")
+        eq(convert, "abc_def")
+        eq(convert, "int123")
+        eq(convert, "_abc")
+        eq(convert, "_123")
+        eq(convert, "__dict__")
+
+    def assert_ascii_equal(self, convert, value):
+        v = convert(value)
+        self.assertEqual(v, value)
+        self.assert_(isinstance(v, str))
+        if have_unicode:
+            unicode_value = unicode(value)
+            v = convert(unicode_value)
+            self.assertEqual(v, value)
+            self.assert_(isinstance(v, str))
+
+    def check_never_namelike(self, convert):
+        raises = self.assertRaises
+        raises(ValueError, convert, "2345")
+        raises(ValueError, convert, "23.45")
+        raises(ValueError, convert, ".45")
+        raises(ValueError, convert, "23.")
+        raises(ValueError, convert, "abc.")
+        raises(ValueError, convert, "-abc")
+        raises(ValueError, convert, "-123")
+        raises(ValueError, convert, "abc-")
+        raises(ValueError, convert, "123-")
+        raises(ValueError, convert, "-")
+        raises(ValueError, convert, ".")
+        raises(ValueError, convert, "&%$*()")
+        raises(ValueError, convert, "")
+
+    def test_datatype_dotted_name(self):
+        convert = self.types.get("dotted-name")
+        raises = self.assertRaises
+        self.check_names(convert)
+        self.check_dotted_names(convert)
+        self.check_never_namelike(convert)
+        raises(ValueError, convert, "abc.")
+        raises(ValueError, convert, ".abc.")
+        raises(ValueError, convert, "abc.def.")
+        raises(ValueError, convert, ".abc.def.")
+        raises(ValueError, convert, ".abc.def")
+
+    def test_datatype_dotted_suffix(self):
+        convert = self.types.get("dotted-suffix")
+        eq = self.assert_ascii_equal
+        raises = self.assertRaises
+        self.check_names(convert)
+        self.check_dotted_names(convert)
+        self.check_never_namelike(convert)
+        eq(convert, ".a")
+        eq(convert, ".a.b")
+        eq(convert, ".a.b.c.d.e.f.g.h.i.j.k.l.m.n.o")
+        raises(ValueError, convert, "abc.")
+        raises(ValueError, convert, ".abc.")
+        raises(ValueError, convert, "abc.def.")
+        raises(ValueError, convert, ".abc.def.")
+
+    def check_dotted_names(self, convert):
+        eq = self.assert_ascii_equal
+        eq(convert, "abc.def")
+        eq(convert, "abc.def.ghi")
+        eq(convert, "a.d.g.g.g.g.g.g.g")
+
+    def test_datatype_inet_address(self):
+        convert = self.types.get("inet-address")
+        eq = self.assertEqual
+        defhost = ZConfig.datatypes.DEFAULT_HOST
+        eq(convert("Host.Example.Com:80"), ("host.example.com", 80))
+        eq(convert(":80"),                 (defhost, 80))
+        eq(convert("80"),                  (defhost, 80))
+        eq(convert("host.EXAMPLE.com"),    ("host.example.com", None))
+        self.assertRaises(ValueError, convert, "40 # foo")
+
+    def test_datatype_integer(self):
+        convert = self.types.get("integer")
+        eq = self.assertEqual
+        raises = self.assertRaises
+
+        eq(convert('-100'), -100)
+        eq(convert('-1'), -1)
+        eq(convert('-0'), 0)
+        eq(convert('0'), 0)
+        eq(convert('1'), 1)
+        eq(convert('100'), 100)
+        eq(convert('65535'), 65535)
+        eq(convert('65536'), 65536)
+
+        big = sys.maxint + 1L  # Python 2.1 needs the L suffix here
+        s = str(big)           # s won't have the suffix
+        eq(convert(s), big)
+        eq(convert("-" + s), -big)
+
+        raises(ValueError, convert, 'abc')
+        raises(ValueError, convert, '-0xabc')
+        raises(ValueError, convert, '')
+        raises(ValueError, convert, '123 456')
+        raises(ValueError, convert, '123-')
+
+    def test_datatype_locale(self):
+        convert = self.types.get("locale")
+        # Python supports "C" even when the _locale module is not available
+        self.assertEqual(convert("C"), "C")
+        self.assertRaises(ValueError, convert, "locale-does-not-exist")
+
+    def test_datatype_port(self):
+        convert = self.types.get("port-number")
+        eq = self.assertEqual
+        raises = self.assertRaises
+
+        raises(ValueError, convert, '-1')
+        raises(ValueError, convert, '0')
+        eq(convert('1'), 1)
+        eq(convert('80'), 80)
+        eq(convert('1023'), 1023)
+        eq(convert('1024'), 1024)
+        eq(convert('60000'), 60000)
+        eq(convert('65535'), 0xffff)
+        raises(ValueError, convert, '65536')
+
+    def test_datatype_socket_address(self):
+        convert = self.types.get("socket-address")
+        eq = self.assertEqual
+        AF_INET = socket.AF_INET
+        defhost = ZConfig.datatypes.DEFAULT_HOST
+
+        def check(value, family, address, self=self, convert=convert):
+            a = convert(value)
+            self.assertEqual(a.family, family)
+            self.assertEqual(a.address, address)
+
+        check("Host.Example.Com:80", AF_INET, ("host.example.com", 80))
+        check(":80",                 AF_INET, (defhost, 80))
+        check("80",                  AF_INET, (defhost, 80))
+        check("host.EXAMPLE.com",    AF_INET, ("host.example.com",None))
+        a1 = convert("/tmp/var/@345.4")
+        a2 = convert("/tmp/var/@345.4:80")
+        self.assertEqual(a1.address, "/tmp/var/@345.4")
+        self.assertEqual(a2.address, "/tmp/var/@345.4:80")
+        if hasattr(socket, "AF_UNIX"):
+            self.assertEqual(a1.family, socket.AF_UNIX)
+            self.assertEqual(a2.family, socket.AF_UNIX)
+        else:
+            self.assert_(a1.family is None)
+            self.assert_(a2.family is None)
+
+    def test_ipaddr_or_hostname(self):
+        convert = self.types.get('ipaddr-or-hostname')
+        eq = self.assertEqual
+        raises = self.assertRaises
+        eq(convert('hostname'),          'hostname')
+        eq(convert('hostname.com'),      'hostname.com')
+        eq(convert('www.hostname.com'),  'www.hostname.com')
+        eq(convert('HOSTNAME'),          'hostname')
+        eq(convert('HOSTNAME.COM'),      'hostname.com')
+        eq(convert('WWW.HOSTNAME.COM'),  'www.hostname.com')
+        eq(convert('127.0.0.1'),         '127.0.0.1')
+        raises(ValueError, convert,  '1hostnamewithleadingnumeric')
+        raises(ValueError, convert,  '255.255')
+        raises(ValueError, convert,  '12345678')
+        raises(ValueError, convert,  '999.999.999.999')
+        raises(ValueError, convert,  'a!badhostname')
+
+    def test_existing_directory(self):
+        convert = self.types.get('existing-directory')
+        eq = self.assertEqual
+        raises = self.assertRaises
+        eq(convert('.'), '.')
+        eq(convert(os.path.dirname(here)), os.path.dirname(here))
+        raises(ValueError, convert, tempfile.mktemp())
+
+    def test_existing_file(self):
+        convert = self.types.get('existing-file')
+        eq = self.assertEqual
+        raises = self.assertRaises
+        eq(convert('.'), '.')
+        eq(convert(here), here)
+        raises(ValueError, convert, tempfile.mktemp())
+
+    def test_existing_path(self):
+        convert = self.types.get('existing-path')
+        eq = self.assertEqual
+        raises = self.assertRaises
+        eq(convert('.'), '.')
+        eq(convert(here), here)
+        eq(convert(os.path.dirname(here)), os.path.dirname(here))
+        raises(ValueError, convert, tempfile.mktemp())
+
+    def test_existing_dirpath(self):
+        convert = self.types.get('existing-dirpath')
+        eq = self.assertEqual
+        raises = self.assertRaises
+        eq(convert('.'), '.')
+        eq(convert(here), here)
+        raises(ValueError, convert, '/a/hopefully/nonexistent/path')
+        raises(ValueError, convert, here + '/bogus')
+
+    def test_byte_size(self):
+        eq = self.assertEqual
+        raises = self.assertRaises
+        convert = self.types.get('byte-size')
+        eq(convert('128'), 128)
+        eq(convert('128KB'), 128*1024)
+        eq(convert('128MB'), 128*1024*1024)
+        eq(convert('128GB'), 128*1024*1024*1024L)
+        raises(ValueError, convert, '128TB')
+        eq(convert('128'), 128)
+        eq(convert('128kb'), 128*1024)
+        eq(convert('128mb'), 128*1024*1024)
+        eq(convert('128gb'), 128*1024*1024*1024L)
+        raises(ValueError, convert, '128tb')
+
+    def test_time_interval(self):
+        eq = self.assertEqual
+        raises = self.assertRaises
+        convert = self.types.get('time-interval')
+        eq(convert('120'), 120)
+        eq(convert('120S'), 120)
+        eq(convert('120M'), 120*60)
+        eq(convert('120H'), 120*60*60)
+        eq(convert('120D'), 120*60*60*24)
+        raises(ValueError, convert, '120W')
+        eq(convert('120'), 120)
+        eq(convert('120s'), 120)
+        eq(convert('120m'), 120*60)
+        eq(convert('120h'), 120*60*60)
+        eq(convert('120d'), 120*60*60*24)
+        raises(ValueError, convert, '120w')
+
+    def test_timedelta(self):
+        eq = self.assertEqual
+        raises = self.assertRaises
+        convert = self.types.get('timedelta')
+        eq(convert('4w'), datetime.timedelta(weeks=4))
+        eq(convert('2d'), datetime.timedelta(days=2))
+        eq(convert('7h'), datetime.timedelta(hours=7))
+        eq(convert('12m'), datetime.timedelta(minutes=12))
+        eq(convert('14s'), datetime.timedelta(seconds=14))
+        eq(convert('4w 2d 7h 12m 14s'),
+           datetime.timedelta(2, 14, minutes=12, hours=7, weeks=4))
+
+
+class RegistryTestCase(unittest.TestCase):
+
+    def test_registry_does_not_mask_toplevel_imports(self):
+        old_sys_path = sys.path[:]
+        tmpdir = tempfile.mkdtemp(prefix="test_datatypes_")
+        fn = os.path.join(tmpdir, "datatypes.py")
+        f = open(fn, "w")
+        f.write(TEST_DATATYPE_SOURCE)
+        f.close()
+        registry = ZConfig.datatypes.Registry()
+
+        # we really want the temp area to override everything else:
+        sys.path.insert(0, tmpdir)
+        try:
+            datatype = registry.get("datatypes.my_sample_datatype")
+        finally:
+            shutil.rmtree(tmpdir)
+            sys.path[:] = old_sys_path
+        self.assertEqual(datatype, 42)
+
+TEST_DATATYPE_SOURCE = """
+# sample datatypes file
+
+my_sample_datatype = 42
+"""
+
+
+def test_suite():
+    suite = unittest.makeSuite(DatatypeTestCase)
+    suite.addTest(unittest.makeSuite(RegistryTestCase))
+    return suite
+
+
+if __name__ == '__main__':
+    unittest.main(defaultTest='test_suite')

trunk/src/ZConfig/tests/test_loader.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Tests of ZConfig.loader classes and helper functions."""
+
+import os.path
+import sys
+import tempfile
+import unittest
+import urllib2
+
+from StringIO import StringIO
+
+import ZConfig
+import ZConfig.loader
+import ZConfig.url
+
+from ZConfig.tests.support import CONFIG_BASE, TestBase
+
+
+try:
+    myfile = __file__
+except NameError:
+    myfile = sys.argv[0]
+
+myfile = os.path.abspath(myfile)
+LIBRARY_DIR = os.path.join(os.path.dirname(myfile), "library")
+
+
+class LoaderTestCase(TestBase):
+
+    def test_schema_caching(self):
+        loader = ZConfig.loader.SchemaLoader()
+        url = ZConfig.url.urljoin(CONFIG_BASE, "simple.xml")
+        schema1 = loader.loadURL(url)
+        schema2 = loader.loadURL(url)
+        self.assert_(schema1 is schema2)
+
+    def test_simple_import_with_cache(self):
+        loader = ZConfig.loader.SchemaLoader()
+        url1 = ZConfig.url.urljoin(CONFIG_BASE, "library.xml")
+        schema1 = loader.loadURL(url1)
+        sio = StringIO("<schema>"
+                       "  <import src='library.xml'/>"
+                       "  <section type='type-a' name='section'/>"
+                       "</schema>")
+        url2 = ZConfig.url.urljoin(CONFIG_BASE, "stringio")
+        schema2 = loader.loadFile(sio, url2)
+        self.assert_(schema1.gettype("type-a") is schema2.gettype("type-a"))
+
+    def test_simple_import_using_prefix(self):
+        self.load_schema_text("""\
+            <schema prefix='ZConfig.tests.library'>
+              <import package='.thing'/>
+            </schema>
+            """)
+
+    def test_import_errors(self):
+        # must specify exactly one of package or src
+        self.assertRaises(ZConfig.SchemaError, ZConfig.loadSchemaFile,
+                          StringIO("<schema><import/></schema>"))
+        self.assertRaises(ZConfig.SchemaError, ZConfig.loadSchemaFile,
+                          StringIO("<schema>"
+                                   "  <import src='library.xml'"
+                                   "          package='ZConfig'/>"
+                                   "</schema>"))
+        # cannot specify src and file
+        self.assertRaises(ZConfig.SchemaError, ZConfig.loadSchemaFile,
+                          StringIO("<schema>"
+                                   "  <import src='library.xml'"
+                                   "          file='other.xml'/>"
+                                   "</schema>"))
+        # cannot specify module as package
+        sio = StringIO("<schema>"
+                       "  <import package='ZConfig.tests.test_loader'/>"
+                       "</schema>")
+        try:
+            ZConfig.loadSchemaFile(sio)
+        except ZConfig.SchemaResourceError, e:
+            self.assertEqual(e.filename, "component.xml")
+            self.assertEqual(e.package, "ZConfig.tests.test_loader")
+            self.assert_(e.path is None)
+            # make sure the str() doesn't raise an unexpected exception
+            str(e)
+        else:
+            self.fail("expected SchemaResourceError")
+
+    def test_import_from_package(self):
+        loader = ZConfig.loader.SchemaLoader()
+        sio = StringIO("<schema>"
+                       "  <import package='ZConfig.tests.library.widget'/>"
+                       "</schema>")
+        schema = loader.loadFile(sio)
+        self.assert_(schema.gettype("widget-a") is not None)
+
+    def test_import_from_package_with_file(self):
+        loader = ZConfig.loader.SchemaLoader()
+        sio = StringIO("<schema>"
+                       "  <import package='ZConfig.tests.library.widget'"
+                       "          file='extra.xml' />"
+                       "</schema>")
+        schema = loader.loadFile(sio)
+        self.assert_(schema.gettype("extra-type") is not None)
+
+    def test_import_from_package_extra_directory(self):
+        loader = ZConfig.loader.SchemaLoader()
+        sio = StringIO("<schema>"
+                       "  <import package='ZConfig.tests.library.thing'"
+                       "          file='extras.xml' />"
+                       "</schema>")
+        schema = loader.loadFile(sio)
+        self.assert_(schema.gettype("extra-thing") is not None)
+
+    def test_import_from_package_with_missing_file(self):
+        loader = ZConfig.loader.SchemaLoader()
+        sio = StringIO("<schema>"
+                       "  <import package='ZConfig.tests.library.widget'"
+                       "          file='notthere.xml' />"
+                       "</schema>")
+        try:
+            loader.loadFile(sio)
+        except ZConfig.SchemaResourceError, e:
+            self.assertEqual(e.filename, "notthere.xml")
+            self.assertEqual(e.package, "ZConfig.tests.library.widget")
+            self.assert_(e.path)
+            # make sure the str() doesn't raise an unexpected exception
+            str(e)
+        else:
+            self.fail("expected SchemaResourceError")
+
+    def test_import_from_package_with_directory_file(self):
+        loader = ZConfig.loader.SchemaLoader()
+        sio = StringIO("<schema>"
+                       "  <import package='ZConfig.tests.library.widget'"
+                       "          file='really/notthere.xml' />"
+                       "</schema>")
+        self.assertRaises(ZConfig.SchemaError, loader.loadFile, sio)
+
+    def test_import_two_components_one_package(self):
+        loader = ZConfig.loader.SchemaLoader()
+        sio = StringIO("<schema>"
+                       "  <import package='ZConfig.tests.library.widget' />"
+                       "  <import package='ZConfig.tests.library.widget'"
+                       "          file='extra.xml' />"
+                       "</schema>")
+        schema = loader.loadFile(sio)
+        schema.gettype("widget-a")
+        schema.gettype("extra-type")
+
+    def test_import_component_twice_1(self):
+        # Make sure we can import a component twice from a schema.
+        # This is most likely to occur when the component is imported
+        # from each of two other components, or from the top-level
+        # schema and a component.
+        loader = ZConfig.loader.SchemaLoader()
+        sio = StringIO("<schema>"
+                       "  <import package='ZConfig.tests.library.widget' />"
+                       "  <import package='ZConfig.tests.library.widget' />"
+                       "</schema>")
+        schema = loader.loadFile(sio)
+        schema.gettype("widget-a")
+
+    def test_import_component_twice_2(self):
+        # Make sure we can import a component from a config file even
+        # if it has already been imported from the schema.
+        loader = ZConfig.loader.SchemaLoader()
+        sio = StringIO("<schema>"
+                       "  <import package='ZConfig.tests.library.widget' />"
+                       "</schema>")
+        schema = loader.loadFile(sio)
+        loader = ZConfig.loader.ConfigLoader(schema)
+        sio = StringIO("%import ZConfig.tests.library.widget")
+        loader.loadFile(sio)
+
+    def test_urlsplit_urlunsplit(self):
+        # Extracted from Python's test.test_urlparse module:
+        for url, parsed, split in [
+            ('http://www.python.org',
+             ('http', 'www.python.org', '', '', '', ''),
+             ('http', 'www.python.org', '', '', '')),
+            ('http://www.python.org#abc',
+             ('http', 'www.python.org', '', '', '', 'abc'),
+             ('http', 'www.python.org', '', '', 'abc')),
+            ('http://www.python.org/#abc',
+             ('http', 'www.python.org', '/', '', '', 'abc'),
+             ('http', 'www.python.org', '/', '', 'abc')),
+            ("http://a/b/c/d;p?q#f",
+             ('http', 'a', '/b/c/d', 'p', 'q', 'f'),
+             ('http', 'a', '/b/c/d;p', 'q', 'f')),
+            ('file:///tmp/junk.txt',
+             ('file', '', '/tmp/junk.txt', '', '', ''),
+             ('file', '', '/tmp/junk.txt', '', '')),
+            ]:
+            result = ZConfig.url.urlsplit(url)
+            self.assertEqual(result, split)
+            result2 = ZConfig.url.urlunsplit(result)
+            self.assertEqual(result2, url)
+
+    def test_file_url_normalization(self):
+        self.assertEqual(
+            ZConfig.url.urlnormalize("file:/abc/def"),
+            "file:///abc/def")
+        self.assertEqual(
+            ZConfig.url.urlunsplit(("file", "", "/abc/def", "", "")),
+            "file:///abc/def")
+        self.assertEqual(
+            ZConfig.url.urljoin("file:/abc/", "def"),
+            "file:///abc/def")
+        self.assertEqual(
+            ZConfig.url.urldefrag("file:/abc/def#frag"),
+            ("file:///abc/def", "frag"))
+
+    def test_isPath(self):
+        assert_ = self.assert_
+        isPath = ZConfig.loader.BaseLoader().isPath
+        assert_(isPath("abc"))
+        assert_(isPath("abc/def"))
+        assert_(isPath("/abc"))
+        assert_(isPath("/abc/def"))
+        assert_(isPath(r"\abc"))
+        assert_(isPath(r"\abc\def"))
+        assert_(isPath(r"c:\abc\def"))
+        assert_(not isPath("http://www.example.com/"))
+        assert_(not isPath("http://www.example.com/sample.conf"))
+        assert_(not isPath("file:///etc/zope/zope.conf"))
+        assert_(not isPath("file:///c|/foo/bar.conf"))
+
+
+class TestNonExistentResources(unittest.TestCase):
+
+    # XXX Not sure if this is the best approach for these.  These
+    # tests make sure that the error reported by ZConfig for missing
+    # resources is handled in a consistent way.  Since ZConfig uses
+    # urllib2.urlopen() for opening all resources, what we do is
+    # replace that function with one that always raises an exception.
+    # Since urllib2.urlopen() can raise either IOError or OSError
+    # (depending on the version of Python), we run test for each
+    # exception.  urllib2.urlopen() is restored after running the
+    # test.
+
+    def setUp(self):
+        self.urllib2_urlopen = urllib2.urlopen
+        urllib2.urlopen = self.fake_urlopen
+
+    def tearDown(self):
+        urllib2.urlopen = self.urllib2_urlopen
+
+    def fake_urlopen(self, url):
+        raise self.error()
+
+    def test_nonexistent_file_ioerror(self):
+        self.error = IOError
+        self.check_nonexistent_file()
+
+    def test_nonexistent_file_oserror(self):
+        self.error = OSError
+        self.check_nonexistent_file()
+
+    def check_nonexistent_file(self):
+        fn = tempfile.mktemp()
+        schema = ZConfig.loadSchemaFile(StringIO("<schema/>"))
+        self.assertRaises(ZConfig.ConfigurationError,
+                          ZConfig.loadSchema, fn)
+        self.assertRaises(ZConfig.ConfigurationError,
+                          ZConfig.loadConfig, schema, fn)
+        self.assertRaises(ZConfig.ConfigurationError,
+                          ZConfig.loadConfigFile, schema,
+                          StringIO("%include " + fn))
+        self.assertRaises(ZConfig.ConfigurationError,
+                          ZConfig.loadSchema,
+                          "http://www.zope.org/no-such-document/")
+        self.assertRaises(ZConfig.ConfigurationError,
+                          ZConfig.loadConfig, schema,
+                          "http://www.zope.org/no-such-document/")
+
+
+def test_suite():
+    suite = unittest.makeSuite(LoaderTestCase)
+    suite.addTest(unittest.makeSuite(TestNonExistentResources))
+    return suite
+
+if __name__ == '__main__':
+    unittest.main(defaultTest='test_suite')

trunk/src/ZConfig/tests/test_schema.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Tests of ZConfig schemas."""
+
+import unittest
+
+import ZConfig
+
+from ZConfig.tests.support import TestBase, CONFIG_BASE
+
+
+def uppercase(value):
+    return str(value).upper()
+
+def appsection(value):
+    return MySection(value)
+
+def get_foo(section):
+    return section.foo
+
+class MySection:
+    def __init__(self, value):
+        self.conf = value
+
+
+def get_section_attributes(section):
+    L = list(section.getSectionAttributes())
+    L.sort()
+    return L
+
+
+class SchemaTestCase(TestBase):
+    """Tests of the basic schema support itself."""
+
+    def test_minimal_schema(self):
+        schema = self.load_schema_text("<schema/>")
+        self.assertEqual(len(schema), 0)
+        self.assertRaises(IndexError,
+                          lambda schema=schema: schema[0])
+        self.assertRaises(ZConfig.ConfigurationError,
+                          schema.getinfo, "foo")
+
+    def test_simple(self):
+        schema, conf = self.load_both("simple.xml", "simple.conf")
+        self._verifySimpleConf(conf)
+
+    def _verifySimpleConf(self,conf):
+        eq = self.assertEqual
+        eq(conf.var1, 'abc')
+        eq(conf.int_var, 12)
+        eq(conf.float_var, 12.02)
+        eq(conf.neg_int, -2)
+
+        check = self.assert_
+        check(conf.true_var_1)
+        check(conf.true_var_2)
+        check(conf.true_var_3)
+        check(not conf.false_var_1)
+        check(not conf.false_var_2)
+        check(not conf.false_var_3)
+
+    def test_app_datatype(self):
+        dtname = __name__ + ".uppercase"
+        schema = self.load_schema_text("""\
+            <schema>
+              <key name='a' datatype='%s'/>
+              <key name='b' datatype='%s' default='abc'/>
+              <multikey name='c' datatype='%s'>
+                <default>abc</default>
+                <default>abc</default>
+                </multikey>
+              <multikey name='d' datatype='%s'>
+                <default>not</default>
+                <default>lower</default>
+                <default>case</default>
+                </multikey>
+            </schema>
+            """ % (dtname, dtname, dtname, dtname))
+        conf = self.load_config_text(schema, """\
+                                     a qwerty
+                                     c upp
+                                     c er
+                                     c case
+                                     """)
+        eq = self.assertEqual
+        eq(conf.a, 'QWERTY')
+        eq(conf.b, 'ABC')
+        eq(conf.c, ['UPP', 'ER', 'CASE'])
+        eq(conf.d, ['NOT', 'LOWER', 'CASE'])
+        eq(get_section_attributes(conf),
+           ["a", "b", "c", "d"])
+
+    def test_app_sectiontype(self):
+        schema = self.load_schema_text("""\
+            <schema datatype='.appsection' prefix='%s'>
+              <sectiontype name='foo' datatype='.MySection'>
+                <key name='sample' datatype='integer' default='345'/>
+                </sectiontype>
+              <section name='sect' type='foo' />
+            </schema>
+            """ % __name__)
+        conf = self.load_config_text(schema, """\
+                                     <foo sect>
+                                       sample 42
+                                     </foo>
+                                     """)
+        self.assert_(isinstance(conf, MySection))
+        o1 = conf.conf.sect
+        self.assert_(isinstance(o1, MySection))
+        self.assertEqual(o1.conf.sample, 42)
+
+    def test_empty_sections(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='section'/>
+              <section type='section' name='s1'/>
+              <section type='section' name='s2'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, """\
+                                     <section s1>
+                                     </section>
+                                     <section s2/>
+                                     """)
+        self.assert_(conf.s1 is not None)
+        self.assert_(conf.s2 is not None)
+        self.assertEqual(get_section_attributes(conf),
+                         ["s1", "s2"])
+
+    def test_deeply_nested_sections(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='type1'>
+                <key name='key' default='type1-value'/>
+              </sectiontype>
+              <sectiontype name='type2'>
+                <key name='key' default='type2-value'/>
+                <section name='sect' type='type1'/>
+              </sectiontype>
+              <sectiontype name='type3'>
+                <key name='key' default='type3-value'/>
+                <section name='sect' type='type2'/>
+              </sectiontype>
+              <section name='sect' type='type3'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, """\
+                                     <type3 sect>
+                                       key sect3-value
+                                       <type2 sect>
+                                         key sect2-value
+                                         <type1 sect/>
+                                       </type2>
+                                     </type3>
+                                     """)
+        eq = self.assertEqual
+        eq(conf.sect.sect.sect.key, "type1-value")
+        eq(conf.sect.sect.key, "sect2-value")
+        eq(conf.sect.key, "sect3-value")
+        eq(get_section_attributes(conf),
+           ["sect"])
+        eq(get_section_attributes(conf.sect),
+           ["key", "sect"])
+        eq(get_section_attributes(conf.sect.sect),
+           ["key", "sect"])
+        eq(get_section_attributes(conf.sect.sect.sect),
+           ["key"])
+
+    def test_multivalued_keys(self):
+        schema = self.load_schema_text("""\
+            <schema handler='def'>
+              <multikey name='a' handler='ABC' />
+              <multikey name='b' datatype='integer'>
+                <default>1</default>
+                <default>2</default>
+              </multikey>
+              <multikey name='c' datatype='integer'>
+                <default>3</default>
+                <default>4</default>
+                <default>5</default>
+              </multikey>
+              <multikey name='d' />
+            </schema>
+            """)
+        conf = self.load_config_text(schema, """\
+                                     a foo
+                                     a bar
+                                     c 41
+                                     c 42
+                                     c 43
+                                     """, num_handlers=2)
+        L = []
+        self.handlers({'abc': L.append,
+                       'DEF': L.append})
+        self.assertEqual(L, [['foo', 'bar'], conf])
+        L = []
+        self.handlers({'abc': None,
+                       'DEF': L.append})
+        self.assertEqual(L, [conf])
+        self.assertEqual(conf.a, ['foo', 'bar'])
+        self.assertEqual(conf.b, [1, 2])
+        self.assertEqual(conf.c, [41, 42, 43])
+        self.assertEqual(conf.d, [])
+        self.assertEqual(get_section_attributes(conf),
+                         ["a", "b", "c", "d"])
+
+    def test_multikey_required(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <multikey name='k' required='yes'/>
+            </schema>
+            """)
+        self.assertRaises(ZConfig.ConfigurationError,
+                          self.load_config_text, schema, "")
+
+    def test_multisection_required(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='s'/>
+              <multisection name='*' attribute='s' type='s' required='yes'/>
+            </schema>
+            """)
+        self.assertRaises(ZConfig.ConfigurationError,
+                          self.load_config_text, schema, "")
+
+    def test_key_required_but_missing(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <key name='k' required='yes'/>
+            </schema>
+            """)
+        self.assertRaises(ZConfig.ConfigurationError,
+                          self.load_config_text, schema, "")
+
+    def test_section_required_but_missing(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='k'/>
+              <section name='k' type='k' required='yes'/>
+            </schema>
+            """)
+        self.assertRaises(ZConfig.ConfigurationError,
+                          self.load_config_text, schema, "")
+
+    def test_key_default_element(self):
+        self.assertRaises(
+            ZConfig.SchemaError, self.load_schema_text, """\
+            <schema>
+              <key name='name'>
+                <default>text</default>
+              </key>
+            </schema>
+            """)
+
+    def test_bad_handler_maps(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <key name='a' handler='abc'/>
+              <key name='b' handler='def'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, """\
+                                     a foo
+                                     b bar
+                                     """, num_handlers=2)
+        self.assertEqual(get_section_attributes(conf),
+                         ["a", "b"])
+        self.assertRaises(ZConfig.ConfigurationError,
+                          self.handlers, {'abc': id, 'ABC': id, 'def': id})
+        self.assertRaises(ZConfig.ConfigurationError,
+                          self.handlers, {})
+
+    def test_handler_ordering(self):
+        schema = self.load_schema_text("""\
+            <schema handler='c'>
+              <sectiontype name='inner'>
+              </sectiontype>
+              <sectiontype name='outer'>
+                <section type='inner' name='sect-inner' handler='a'/>
+              </sectiontype>
+              <section type='outer' name='sect-outer' handler='b'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, """\
+                                     <outer sect-outer>
+                                       <inner sect-inner/>
+                                     </outer>
+                                     """, num_handlers=3)
+        L = []
+        self.handlers({'a': L.append,
+                       'b': L.append,
+                       'c': L.append})
+        outer = conf.sect_outer
+        inner = outer.sect_inner
+        self.assertEqual(L, [inner, outer, conf])
+
+    def test_duplicate_section_names(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='sect'/>
+              <sectiontype name='nesting'>
+                <section name='a' type='sect'/>
+              </sectiontype>
+              <section name='a' type='nesting'/>
+            </schema>
+            """)
+        self.assertRaises(ZConfig.ConfigurationError, self.load_config_text,
+                          schema, """\
+                          <sect a/>
+                          <sect a/>
+                          """)
+        conf = self.load_config_text(schema, """\
+                                     <nesting a>
+                                       <sect a/>
+                                     </nesting>
+                                     """)
+
+    def test_disallowed_duplicate_attribute(self):
+        self.assertRaises(ZConfig.SchemaError, self.load_schema_text, """\
+                          <schema>
+                            <key name='a'/>
+                            <key name='b' attribute='a'/>
+                          </schema>
+                          """)
+
+    def test_unknown_datatype_name(self):
+        self.assertRaises(ZConfig.SchemaError,
+                          self.load_schema_text, "<schema datatype='foobar'/>")
+
+    def test_load_abstracttype(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <abstracttype name='group'>
+                <description>This is an abstract section type.</description>
+              </abstracttype>
+              <sectiontype name='t1' implements='group'>
+                <key name='k1' default='default1'/>
+              </sectiontype>
+              <sectiontype name='t2' implements='group'>
+                <key name='k2' default='default2'/>
+              </sectiontype>
+              <multisection name='*' type='group' attribute='g'/>
+            </schema>
+            """)
+        # check the types that get defined
+        t = schema.gettype("group")
+        self.assert_(t.isabstract())
+        t1 = schema.gettype("t1")
+        self.assert_(not t1.isabstract())
+        self.assert_(t.getsubtype("t1") is t1)
+        t2 = schema.gettype("t2")
+        self.assert_(not t2.isabstract())
+        self.assert_(t.getsubtype("t2") is t2)
+        self.assertRaises(ZConfig.ConfigurationError, t.getsubtype, "group")
+        self.assert_(t1 is not t2)
+        # try loading a config that relies on this schema
+        conf = self.load_config_text(schema, """\
+                                     <t1/>
+                                     <t1>
+                                       k1 value1
+                                     </t1>
+                                     <t2/>
+                                     <t2>
+                                       k2 value2
+                                     </t2>
+                                     """)
+        eq = self.assertEqual
+        eq(get_section_attributes(conf), ["g"])
+        eq(len(conf.g), 4)
+        eq(conf.g[0].k1, "default1")
+        eq(conf.g[1].k1, "value1")
+        eq(conf.g[2].k2, "default2")
+        eq(conf.g[3].k2, "value2")
+
+        # white box:
+        self.assert_(conf.g[0].getSectionDefinition() is t1)
+        self.assert_(conf.g[1].getSectionDefinition() is t1)
+        self.assert_(conf.g[2].getSectionDefinition() is t2)
+        self.assert_(conf.g[3].getSectionDefinition() is t2)
+
+    def test_abstracttype_extension(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <abstracttype name='group'/>
+              <sectiontype name='extra' implements='group'/>
+              <section name='thing' type='group'/>
+            </schema>
+            """)
+        abstype = schema.gettype("group")
+        self.assert_(schema.gettype("extra") is abstype.getsubtype("extra"))
+
+        # make sure we can use the extension in a config:
+        conf = self.load_config_text(schema, "<extra thing/>")
+        self.assertEqual(conf.thing.getSectionType(), "extra")
+        self.assertEqual(get_section_attributes(conf), ["thing"])
+        self.assertEqual(get_section_attributes(conf.thing), [])
+
+    def test_abstracttype_extension_errors(self):
+        # specifying a non-existant abstracttype
+        self.assertRaises(ZConfig.SchemaError, self.load_schema_text, """\
+                          <schema>
+                            <sectiontype name='s' implements='group'/>
+                          </schema>
+                          """)
+        # specifying something that isn't an abstracttype
+        self.assertRaises(ZConfig.SchemaError, self.load_schema_text, """\
+                          <schema>
+                            <sectiontype name='t1'/>
+                            <sectiontype name='t2' implements='t1'/>
+                          </schema>
+                          """)
+
+    def test_arbitrary_key(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <key name='+' required='yes' attribute='keymap'
+                   datatype='integer'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, "some-key 42")
+        self.assertEqual(conf.keymap, {'some-key': 42})
+        self.assertEqual(get_section_attributes(conf), ["keymap"])
+
+    def test_arbitrary_multikey_required(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <multikey name='+' required='yes' attribute='keymap'
+                        datatype='integer'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, """\
+                                     some-key 42
+                                     some-key 43
+                                     """)
+        self.assertEqual(conf.keymap, {'some-key': [42, 43]})
+
+    def test_arbitrary_multikey_optional(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='sect'>
+                <multikey name='+' attribute='keymap'/>
+              </sectiontype>
+              <section name='+' type='sect' attribute='stuff'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, """\
+                                     <sect foo>
+                                       some-key 42
+                                       some-key 43
+                                     </sect>
+                                     """)
+        self.assertEqual(conf.stuff.keymap, {'some-key': ['42', '43']})
+        self.assertEqual(get_section_attributes(conf), ["stuff"])
+
+    def test_arbitrary_multikey_optional_empty(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='sect'>
+                <multikey name='+' attribute='keymap'/>
+              </sectiontype>
+              <section name='+' type='sect' attribute='stuff'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, "<sect foo/>")
+        self.assertEqual(conf.stuff.keymap, {})
+
+    def test_arbitrary_multikey_with_defaults(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <multikey name='+' attribute='keymap'>
+                <default key='a'>value-a1</default>
+                <default key='a'>value-a2</default>
+                <default key='b'>value-b</default>
+              </multikey>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, "")
+        self.assertEqual(conf.keymap, {'a': ['value-a1', 'value-a2'],
+                                       'b': ['value-b']})
+
+    def test_arbitrary_multikey_with_unkeyed_default(self):
+        self.assertRaises(ZConfig.SchemaError,
+                          self.load_schema_text, """\
+                          <schema>
+                            <multikey name='+' attribute='keymap'>
+                              <default>value-a1</default>
+                            </multikey>
+                          </schema>
+                          """)
+
+    def test_arbitrary_key_with_defaults(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <key name='+' attribute='keymap'>
+                <default key='a'>value-a</default>
+                <default key='b'>value-b</default>
+              </key>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, "")
+        self.assertEqual(conf.keymap, {'a': 'value-a', 'b': 'value-b'})
+
+    def test_arbitrary_key_with_unkeyed_default(self):
+        self.assertRaises(ZConfig.SchemaError,
+                          self.load_schema_text, """\
+                          <schema>
+                            <key name='+' attribute='keymap'>
+                              <default>value-a1</default>
+                            </key>
+                          </schema>
+                          """)
+
+    def test_arbitrary_keys_with_others(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <key name='k1' default='v1'/>
+              <key name='k2' default='2' datatype='integer'/>
+              <key name='+' required='yes' attribute='keymap'
+                   datatype='integer'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, """\
+                                     some-key 42
+                                     k2 3
+                                     """)
+        self.assertEqual(conf.k1, 'v1')
+        self.assertEqual(conf.k2, 3)
+        self.assertEqual(conf.keymap, {'some-key': 42})
+        self.assertEqual(get_section_attributes(conf),
+                         ["k1", "k2", "keymap"])
+
+    def test_arbitrary_key_missing(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <key name='+' required='yes' attribute='keymap' />
+            </schema>
+            """)
+        self.assertRaises(ZConfig.ConfigurationError,
+                          self.load_config_text, schema, "# empty config file")
+
+    def test_arbitrary_key_bad_schema(self):
+        self.assertRaises(ZConfig.SchemaError, self.load_schema_text, """\
+                          <schema>
+                            <key name='+' attribute='attr1'/>
+                            <key name='+' attribute='attr2'/>
+                          </schema>
+                          """)
+
+    def test_getrequiredtypes(self):
+        schema = self.load_schema("library.xml")
+        self.assertEqual(schema.getrequiredtypes(), [])
+
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='used'/>
+              <sectiontype name='unused'/>
+              <section type='used' name='a'/>
+            </schema>
+            """)
+        L = schema.getrequiredtypes()
+        L.sort()
+        self.assertEqual(L, ["used"])
+
+    def test_getunusedtypes(self):
+        schema = self.load_schema("library.xml")
+        L = schema.getunusedtypes()
+        L.sort()
+        self.assertEqual(L, ["type-a", "type-b"])
+
+        schema = self.load_schema_text("""\
+            <schema type='top'>
+              <sectiontype name='used'/>
+              <sectiontype name='unused'/>
+              <section type='used' name='a'/>
+            </schema>
+            """)
+        self.assertEqual(schema.getunusedtypes(), ["unused"])
+
+    def test_section_value_mutation(self):
+        schema, conf = self.load_both("simple.xml", "simple.conf")
+        orig = conf.empty
+        new = []
+        conf.empty = new
+        self.assert_(conf.empty is new)
+
+    def test_simple_anonymous_section(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='sect'>
+                <key name='key' default='value'/>
+              </sectiontype>
+              <section name='*' type='sect' attribute='attr'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, "<sect/>")
+        self.assertEqual(conf.attr.key, "value")
+
+    def test_simple_anynamed_section(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='sect'>
+                <key name='key' default='value'/>
+              </sectiontype>
+              <section name='+' type='sect' attribute='attr'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, "<sect name/>")
+        self.assertEqual(conf.attr.key, "value")
+        self.assertEqual(conf.attr.getSectionName(), "name")
+
+        # if we omit the name, it's an error
+        self.assertRaises(ZConfig.ConfigurationError,
+                          self.load_config_text, schema, "<sect/>")
+
+    def test_nested_abstract_sectiontype(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <abstracttype name='abstract'/>
+              <sectiontype name='t1' implements='abstract'/>
+              <sectiontype name='t2' implements='abstract'>
+                <section type='abstract' name='s1'/>
+              </sectiontype>
+              <section type='abstract' name='*' attribute='s2'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, """\
+                                     <t2>
+                                       <t1 s1/>
+                                     </t2>
+                                     """)
+
+    def test_reserved_attribute_prefix(self):
+        template = """\
+            <schema>
+              <sectiontype name='s'/>
+                %s
+            </schema>
+            """
+        def check(thing, self=self, template=template):
+            text = template % thing
+            self.assertRaises(ZConfig.SchemaError,
+                              self.load_schema_text, text)
+
+        check("<key name='a' attribute='getSection'/>")
+        check("<key name='a' attribute='getSectionThing'/>")
+        check("<multikey name='a' attribute='getSection'/>")
+        check("<multikey name='a' attribute='getSectionThing'/>")
+        check("<section type='s' name='*' attribute='getSection'/>")
+        check("<section type='s' name='*' attribute='getSectionThing'/>")
+        check("<multisection type='s' name='*' attribute='getSection'/>")
+        check("<multisection type='s' name='*' attribute='getSectionThing'/>")
+
+    def test_sectiontype_as_schema(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='s'>
+                <key name='skey' default='skey-default'/>
+              </sectiontype>
+              <sectiontype name='t'>
+                <key name='tkey' default='tkey-default'/>
+                <section name='*' type='s' attribute='section'/>
+              </sectiontype>
+            </schema>
+            """)
+        t = schema.gettype("t")
+        conf = self.load_config_text(t, "<s/>")
+        self.assertEqual(conf.tkey, "tkey-default")
+        self.assertEqual(conf.section.skey, "skey-default")
+        self.assertEqual(get_section_attributes(conf), ["section", "tkey"])
+        self.assertEqual(get_section_attributes(conf.section), ["skey"])
+
+    def test_datatype_conversion_error(self):
+        schema_url = "file:///tmp/fake-url-1.xml"
+        config_url = "file:///tmp/fake-url-2.xml"
+        schema = self.load_schema_text("""\
+             <schema>
+               <key name='key' default='bogus' datatype='integer'/>
+             </schema>
+             """, url=schema_url)
+        e = self.get_data_conversion_error(
+            schema, "", config_url)
+        self.assertEqual(e.url, schema_url)
+        self.assertEqual(e.lineno, 2)
+
+        e = self.get_data_conversion_error(schema, """\
+                                           # comment
+
+                                           key splat
+                                           """, config_url)
+        self.assertEqual(e.url, config_url)
+        self.assertEqual(e.lineno, 3)
+
+    def get_data_conversion_error(self, schema, src, url):
+        try:
+            self.load_config_text(schema, src, url=url)
+        except ZConfig.DataConversionError, e:
+            return e
+        else:
+            self.fail("expected ZConfig.DataConversionError")
+
+    def test_numeric_section_name(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='sect'/>
+              <multisection name='*' type='sect' attribute='things'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, "<sect 1 />")
+        self.assertEqual(len(conf.things), 1)
+
+    def test_sectiontype_extension(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='t1'>
+                <key name='k1'/>
+              </sectiontype>
+              <sectiontype name='t2' extends='t1'>
+                <key name='k2'/>
+              </sectiontype>
+              <section name='s' type='t2'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, """\
+                                     <t2 s>
+                                       k1 k1-value
+                                       k2 k2-value
+                                     </t2>
+                                     """)
+        eq = self.assertEqual
+        eq(conf.s.k1, "k1-value")
+        eq(conf.s.k2, "k2-value")
+        eq(get_section_attributes(conf), ["s"])
+        eq(get_section_attributes(conf.s), ["k1", "k2"])
+
+    def test_sectiontype_extension_errors(self):
+        # cannot override key from base
+        self.assertRaises(ZConfig.SchemaError, self.load_schema_text, """\
+                          <schema>
+                            <sectiontype name='t1'>
+                              <key name='k1'/>
+                            </sectiontype>
+                            <sectiontype name='t2' extends='t1'>
+                              <key name='k1'/>
+                            </sectiontype>
+                          </schema>
+                          """)
+        # cannot extend non-existing section
+        self.assertRaises(ZConfig.SchemaError, self.load_schema_text, """\
+                          <schema>
+                            <sectiontype name='t2' extends='t1'/>
+                          </schema>
+                          """)
+        # cannot extend abstract type
+        self.assertRaises(ZConfig.SchemaError, self.load_schema_text, """\
+                          <schema>
+                            <abstracttype name='t1'/>
+                            <sectiontype name='t2' extends='t1'/>
+                          </schema>
+                          """)
+
+    def test_sectiontype_derived_keytype(self):
+        # make sure that a derived section type inherits the keytype
+        # of its base
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='sect' keytype='identifier'/>
+              <sectiontype name='derived' extends='sect'>
+                <key name='foo' attribute='foo'/>
+                <key name='Foo' attribute='Foo'/>
+              </sectiontype>
+              <section name='foo' type='derived'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, """\
+            <derived foo>
+              foo bar
+              Foo BAR
+            </derived>
+            """)
+        self.assertEqual(conf.foo.foo, "bar")
+        self.assertEqual(conf.foo.Foo, "BAR")
+        self.assertEqual(get_section_attributes(conf.foo), ["Foo", "foo"])
+
+    def test_sectiontype_override_keytype(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='base' keytype='identifier' >
+                <key name='+' attribute='map' />
+              </sectiontype>
+              <sectiontype name='derived' keytype='ipaddr-or-hostname'
+                           extends='base' />
+              <section name='*' type='base' attribute='base' />
+              <section name='*' type='derived' attribute='derived' />
+            </schema>
+            """)
+        conf = self.load_config_text(schema, """\
+            <base>
+              ident1 foo
+              Ident2 bar
+            </base>
+            <derived>
+              EXAMPLE.COM foo
+            </derived>
+            """)
+        L = conf.base.map.items()
+        L.sort()
+        self.assertEqual(L, [("Ident2", "bar"), ("ident1", "foo")])
+        L = conf.derived.map.items()
+        L.sort()
+        self.assertEqual(L, [("example.com", "foo")])
+        self.assertEqual(get_section_attributes(conf), ["base", "derived"])
+
+    def test_keytype_applies_to_default_key(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='sect'>
+                <key name='+' attribute='mapping'>
+                  <default key='foo'>42</default>
+                  <default key='BAR'>24</default>
+                </key>
+              </sectiontype>
+              <section type='sect' name='*' attribute='sect'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, "<sect/>")
+        items = conf.sect.mapping.items()
+        items.sort()
+        self.assertEqual(items, [("bar", "24"), ("foo", "42")])
+
+    def test_duplicate_default_key_checked_in_schema(self):
+        self.assertRaises(ZConfig.SchemaError,
+                          self.load_schema_text, """\
+            <schema>
+              <sectiontype name='sect'>
+                <key name='+' attribute='mapping'>
+                  <default key='foo'>42</default>
+                  <default key='Foo'>24</default>
+                </key>
+              </sectiontype>
+              <section type='sect' name='*' attribute='sect'/>
+            </schema>
+            """)
+
+    def test_default_keys_rechecked_clash_in_derived_sectiontype(self):
+        # If the default values associated with a <key name="+"> can't
+        # be supported by a new keytype for a derived sectiontype, an
+        # error should be indicated.
+        self.assertRaises(ZConfig.SchemaError,
+                          self.load_schema_text, """\
+            <schema>
+              <sectiontype name='base' keytype='identifier'>
+                <key name='+' attribute='mapping'>
+                  <default key='foo'>42</default>
+                  <default key='Foo'>42</default>
+                </key>
+              </sectiontype>
+              <sectiontype name='sect' keytype='basic-key'
+                           extends='base'>
+                <!-- should cry foul here -->
+              </sectiontype>
+              <section type='sect' name='*' attribute='sect'/>
+            </schema>
+            """)
+
+    def test_default_keys_rechecked_dont_clash_in_derived_sectiontype(self):
+        # If the default values associated with a <key name="+"> can't
+        # be supported by a new keytype for a derived sectiontype, an
+        # error should be indicated.
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='base' keytype='identifier'>
+                <multikey name='+' attribute='mapping'>
+                  <default key='foo'>42</default>
+                  <default key='Foo'>42</default>
+                </multikey>
+              </sectiontype>
+              <sectiontype name='sect' keytype='basic-key'
+                           extends='base'>
+                <!-- should cry foul here -->
+              </sectiontype>
+              <section type='base' name='*' attribute='base'/>
+              <section type='sect' name='*' attribute='sect'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, """\
+            <base/>
+            <sect/>
+            """)
+        base = conf.base.mapping.items()
+        base.sort()
+        self.assertEqual(base, [("Foo", ["42"]), ("foo", ["42"])])
+        sect = conf.sect.mapping.items()
+        sect.sort()
+        self.assertEqual(sect, [("foo", ["42", "42"])])
+
+    def test_sectiontype_inherited_datatype(self):
+        schema = self.load_schema_text("""\
+            <schema prefix='ZConfig.tests.test_schema'>
+              <sectiontype name='base' datatype='.get_foo'>
+                <key name="foo"/>
+              </sectiontype>
+              <sectiontype name='derived' extends='base'/>
+              <section name='*' type='derived' attribute='splat'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, """\
+            <derived>
+              foo bar
+            </derived>
+            """)
+        self.assertEqual(conf.splat, "bar")
+
+    def test_schema_keytype(self):
+        schema = self.load_schema_text("""\
+            <schema keytype='ipaddr-or-hostname'>
+              <key name='+' attribute='table' datatype='ipaddr-or-hostname'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema,
+                                     "host.example.com 127.0.0.1\n"
+                                     "www.example.org 127.0.0.2\n")
+        table = conf.table
+        self.assertEqual(len(table), 2)
+        L = table.items()
+        L.sort()
+        self.assertEqual(L, [("host.example.com", "127.0.0.1"),
+                             ("www.example.org", "127.0.0.2")])
+
+        self.assertRaises(ZConfig.ConfigurationError,
+                          self.load_config_text, schema, "abc.  127.0.0.1")
+
+    def test_keytype_identifier(self):
+        schema = self.load_schema_text("""\
+           <schema keytype='identifier'>
+             <key name='foo' attribute='foo'/>
+             <key name='Foo' attribute='Foo'/>
+           </schema>
+           """)
+        conf = self.load_config_text(schema,
+                                     "Foo Foo-value\n"
+                                     "foo foo-value\n")
+        self.assertEqual(conf.foo, "foo-value")
+        self.assertEqual(conf.Foo, "Foo-value")
+        self.assertEqual(get_section_attributes(conf), ["Foo", "foo"])
+        # key mis-match based on case:
+        self.assertRaises(ZConfig.ConfigurationError,
+                          self.load_config_text, schema, "FOO frob\n")
+        # attribute names conflict, since the keytype isn't used to
+        # generate attribute names
+        self.assertRaises(ZConfig.SchemaError,
+                          self.load_schema_text, """\
+                          <schema keytype='identifier'>
+                            <key name='foo'/>
+                            <key name='Foo'/>
+                          </schema>
+                          """)
+
+    def test_datatype_casesensitivity(self):
+        self.load_schema_text("<schema datatype='NULL'/>")
+
+    def test_simple_extends(self):
+        schema = self.load_schema_text("""\
+           <schema extends='%s/simple.xml %s/library.xml'>
+             <section name='A' type='type-a' />
+           </schema>
+           """ % (CONFIG_BASE, CONFIG_BASE))
+        self._verifySimpleConf(self.load_config(schema, "simple.conf"))
+
+    def test_extends_fragment_failure(self):
+        self.assertRaises(ZConfig.SchemaError,
+                          self.load_schema_text,
+            "<schema extends='%s/library.xml#foo'/>" % CONFIG_BASE)
+
+    def test_multi_extends_implicit_OK(self):
+        self.load_schema_text("""\
+           <schema extends='%s/base.xml %s/library.xml'>
+             <section name='A' type='type-a' />
+             <section name='X' type='type-X' />
+           </schema>
+           """ % (CONFIG_BASE, CONFIG_BASE))
+
+    def test_multi_extends_explicit_datatype_OK(self):
+        self.load_schema_text("""\
+           <schema extends='%s/base-datatype1.xml %s/base-datatype2.xml'
+                   datatype='null'>
+             <section name='One' type='type-1' />
+             <section name='Two' type='type-2' />
+           </schema>
+           """ % (CONFIG_BASE, CONFIG_BASE))
+
+    def test_multi_extends_explicit_keytype_OK(self):
+        self.load_schema_text("""\
+           <schema extends='%s/base-keytype1.xml %s/base-keytype2.xml'
+                   keytype='%s.uppercase'>
+             <section name='One' type='type-1' />
+             <section name='Two' type='type-2' />
+           </schema>
+           """ % (CONFIG_BASE, CONFIG_BASE, __name__))
+
+    def test_multi_extends_datatype_conflict(self):
+        self.assertRaises(ZConfig.SchemaError,
+                          self.load_schema_text, """\
+            <schema extends='%s/base-datatype1.xml %s/base-datatype2.xml'/>
+            """ % (CONFIG_BASE, CONFIG_BASE))
+
+    def test_multi_extends_keytype_conflict(self):
+        self.assertRaises(ZConfig.SchemaError,
+                          self.load_schema_text, """\
+            <schema extends='%s/base-keytype1.xml %s/base-keytype2.xml'/>
+            """ % (CONFIG_BASE, CONFIG_BASE))
+
+    def test_multiple_descriptions_is_error(self):
+        self.assertRaises(ZConfig.SchemaError,
+                          self.load_schema_text, """\
+            <schema>
+              <description>  foo  </description>
+              <description>  bar  </description>
+            </schema>
+            """)
+
+
+def test_suite():
+    return unittest.makeSuite(SchemaTestCase)
+
+if __name__ == '__main__':
+    unittest.main(defaultTest='test_suite')

trunk/src/ZConfig/tests/test_subst.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Tests of the string interpolation module."""
+
+# This is needed to support Python 2.1.
+from __future__ import nested_scopes
+
+import unittest
+
+from ZConfig import SubstitutionReplacementError, SubstitutionSyntaxError
+from ZConfig.substitution import isname, substitute
+
+
+class SubstitutionTestCase(unittest.TestCase):
+    def test_simple_names(self):
+        d = {"name": "value",
+             "name1": "abc",
+             "name_": "def",
+             "_123": "ghi"}
+        def check(s, v):
+            self.assertEqual(substitute(s, d), v)
+        check("$name", "value")
+        check(" $name ", " value ")
+        check("${name}", "value")
+        check(" ${name} ", " value ")
+        check("$name$name", "valuevalue")
+        check("$name1$name", "abcvalue")
+        check("$name_$name", "defvalue")
+        check("$_123$name", "ghivalue")
+        check("$name $name", "value value")
+        check("$name1 $name", "abc value")
+        check("$name_ $name", "def value")
+        check("$_123 $name", "ghi value")
+        check("splat", "splat")
+        check("$$", "$")
+        check("$$$name$$", "$value$")
+
+    def test_undefined_names(self):
+        d = {"name": "value"}
+        self.assertRaises(SubstitutionReplacementError,
+                          substitute, "$splat", d)
+        self.assertRaises(SubstitutionReplacementError,
+                          substitute, "$splat1", d)
+        self.assertRaises(SubstitutionReplacementError,
+                          substitute, "$splat_", d)
+
+    def test_syntax_errors(self):
+        d = {"name": "${next"}
+        def check(s):
+            self.assertRaises(SubstitutionSyntaxError,
+                              substitute, s, d)
+        check("${")
+        check("${name")
+        check("${1name}")
+        check("${ name}")
+
+    def test_edge_cases(self):
+        # It's debatable what should happen for these cases, so we'll
+        # follow the lead of the Bourne shell here.
+        def check(s):
+            self.assertRaises(SubstitutionSyntaxError,
+                              substitute, s, {})
+        check("$1")
+        check("$")
+        check("$ stuff")
+
+    def test_non_nesting(self):
+        d = {"name": "$value"}
+        self.assertEqual(substitute("$name", d), "$value")
+
+    def test_isname(self):
+        self.assert_(isname("abc"))
+        self.assert_(isname("abc_def"))
+        self.assert_(isname("_abc"))
+        self.assert_(isname("abc_"))
+        self.assert_(not isname("abc-def"))
+        self.assert_(not isname("-def"))
+        self.assert_(not isname("abc-"))
+        self.assert_(not isname(""))
+
+
+def test_suite():
+    return unittest.makeSuite(SubstitutionTestCase)
+
+if __name__ == '__main__':
+    unittest.main(defaultTest='test_suite')

trunk/src/ZConfig/url.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""urlparse-like helpers that normalize file: URLs.
+
+ZConfig and urllib2 expect file: URLs to consistently use the '//'
+hostpart seperator; the functions here enforce this constraint.
+"""
+
+import urlparse as _urlparse
+
+try:
+    from urlparse import urlsplit
+except ImportError:
+    def urlsplit(url):
+        # Check for the fragment here, since Python 2.1.3 didn't get
+        # it right for things like "http://www.python.org#frag".
+        if '#' in url:
+            url, fragment = url.split('#', 1)
+        else:
+            fragment = ''
+        parts = list(_urlparse.urlparse(url))
+        parts[-1] = fragment
+        param = parts.pop(3)
+        if param:
+            parts[2] += ";" + param
+        return tuple(parts)
+
+
+def urlnormalize(url):
+    lc = url.lower()
+    if lc.startswith("file:/") and not lc.startswith("file:///"):
+        url = "file://" + url[5:]
+    return url
+
+
+def urlunsplit(parts):
+    parts = list(parts)
+    parts.insert(3, '')
+    url = _urlparse.urlunparse(tuple(parts))
+    if (parts[0] == "file"
+        and url.startswith("file:/")
+        and not url.startswith("file:///")):
+        url = "file://" + url[5:]
+    return url
+
+
+def urldefrag(url):
+    url, fragment = _urlparse.urldefrag(url)
+    return urlnormalize(url), fragment
+
+
+def urljoin(base, relurl):
+    url = _urlparse.urljoin(base, relurl)
+    if url.startswith("file:/") and not url.startswith("file:///"):
+        url = "file://" + url[5:]
+    return url