Full review of packaging, with multi-eggs setup.py

- multi-eggs setup.py because we really don't want to split the repository
  in several ones
- move 'components.xml' since namespace packages should not contain anything
- workaround python-mock.sf.net because not eggified:
  automatically get a copy of mock.py (license: BSD) when setting up 'neotests'

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2750 71dcc9de-d417-0410-9af5-da40c76e7ee4

MANIFEST.in

-graft tools
-include neoadmin neoctl neomaster neomigrate neostorage neo.conf README TODO
-global-exclude *~

README

@@ -9,6 +9,8 @@ Requirements
 
     - For python 2.4: ctypes http://python.net/crew/theller/ctypes/
       (packaged with later python versions)
+      Note that setup.py does not define any dependency to 'ctypes' so you will
+      have to install it explicitely.
 
     - MySQLdb http://sourceforge.net/projects/mysql-python
 
@@ -71,7 +73,7 @@ How to use
       replacing its filestorage subsection by a NEOStorage one.
       It should look like :
 
-      %import neo
+      %import neo.client
       <zodb_db main>
           # Main FileStorage database
           <NEOStorage>

buildout/INSTALL.txt

-##########################################
-########### HOW TO INSTALL NEO ###########
-##########################################
-
-
-###########
-## Intro ##
-###########
-
-Welcome in the neo installation tutorial. This will learn to you how to install
-and configure neo on your own computer.
-There's currently three types of possible installations
-
-- Quick installation from buildout (1)
-- Installation from source (3 then 1)
-
-#############
-## Summary ##
-#############
-
-0. Requirements
-1. Quick installation
-2. Running tests
-3. Building neo eggs
-4. Common errors
-
-######################
-### 0.Requirements ###
-######################
-
-In order to install neo services on your computer, you'll need the following
-requirements :
-
-- Linux 2.6 or higher ( This tutorial has been tested for Mandriva 2010 )
-- python2.4 or higher
-- ctypes (http://python.net/crew/theller/ctypes/) (If you're using python2.5 or
-higher, ctypes is directly included with python)
-- mock (http://python-mock.sourceforge.net/) (Only if you plan to make the
-neo tests)
-
-############################
-### 1.Quick installation ###
-############################
-
-You're a neo-newbie ? You don't need neo sources ? You want a full version and
-not just one or two modules ? Or, more commonly, you're in a hurry ? So type
-the following commands :
-
-$ svn checkout https//svn.erp5.org/repos/neo/trunk/buildout/software-profiles/neo.cfg
-When your bootstrap's done, check that you've got bin/buildout script. Then
-Edit your buildout.cfg file, and delete all the neo-eggs which corresponds
-to the parts you don't need. Then run :
-
-$ cd neo/trunk/eggs
-$ ./makeeggs
-$ cd ../buildout/software-profiles
-$ python -S -c 'import urllib;print urllib.urlopen("http://svn.zope.org/' \
-'*checkout*''/zc.buildout/trunk/bootstrap/bootstrap.py").read()' | python -S -
-
-$ bin/buildout
-
-Congratulations, neo's installation is over ! You can now launch the different
-services you installed with typing bin/<neo_script>.
-
-#########################
-### 2. Running tests ###
-#########################
-
-This part is recommended for experienced users, who already installed all other
-neo parts, and could manage neo pretty easily.
-
-Export your path to set up your environment:
-
-$ export $PATH=$PATH:<download_neo>/trunk/bin
-Add it to your neo installation :
-
-$ bin/buildout -Nvv
-
-Run neo tests :
-
-$ bin/neotestrunner -fzu (functional, unit tests and zodb tests)
-
-If this returns no errors or fails, it means that it's over for you ! Neo's
-correctly installed on your machine, congratulations !
-
-#############################
-### 3. Building neo eggs ###
-#############################
-
-This part is recommended for experienced users, who wants to make a neo
-installation from beginning. It explains how to make eggs from a
-precise neo part, instead of eggifying the whole project.
-
-Each part of neo (lib, admin, master, storage, neoctl, client) has its own
-setup.py file, which is located on <download_neo>/trunk/setup_files, ans which
-allows to build the linked egg. See VI. Setup files to check which setup.py
-file you need. Then execute the following commands :
-
-$ svn checkout https://svn.erp5.org/repos/neo/trunk
-$ cd <download_neo>/trunk/neo
-(setup_cmn.py is the main part of neo, needed by all other processes)
-$ python setup.*.py bdist_egg
-
-Check in your trunk/bdist folder that you have an egg for each part of neo you
-want to install. If it's ok, then go on to part II.
-
-########################
-### 4. Common errors ###
-########################
-
-Problem : The egg I need is not present in my trunk/bdist folder.
-Solution : Select the correct setup_<part>.py in trunk/setup_files
-
-Problem: bin/buildout is not present
-Solution: Launch the bootstrap command
-
-Problem : bootstrap command is not working
-Solution : Try to use another python version
-
-Tests errors :
-  'neoadmin' not found
-  'neomaster' not found
-  'neostorage' not found
-Solution :
-  Check your path and remember to export <download_neo>/trunk/bin to it
-

buildout/eggification_mock.txt

-How to eggify Mock projet (Neo dependency)
-
-- Download source http://python-mock.sourceforge.net/ in a mock/ folder
-- Write setup.py file
-
-from setuptools import setup, find_packages
-
-setup(
-    name='mock',
-    version='1.0',
-    description='Manipulate mock objects for NEO',
-    author='Dave Kirby',
-    author_email='test@admin.com',
-    url='http://python-mock.sourceforge.net/',
-    license="BSD",
-    packages=find_packages(),
-    package_dir={'mock': 'mock'},
-    zip_safe=False,
-)
-
-- Launch python setup.py bdist_egg
-
-The created egg is in dist/ folder.

buildout/software-profiles/neo.cfg

-#Neo global config file - partitioned
-
-[buildout]
-# extends-cache = extends-cache
-parts =
-  neoinstall
-find-links =
-  ../../eggs/
-
-[neoinstall]
-recipe = zc.recipe.egg
-interpreter = python
-scripts =
-  neotestrunner
-  neomigrate
-  neomaster
-  neoadmin
-  neostorage
-  neoctl
-eggs =
-  neotests
-  neoadmin
-  neoctl
-  neostorage
-  neoclient
-  neomaster
-  # for unit tests
-  zope.testing
-  psutil

eggs/makeeggs

-#!/bin/sh
-PYTHON=python2.6
-INSTALL_DIR=eggs/
-
-cd dirname $0/..
-case "$1" in
-  "develop")
-  MODE=develop
-  PYTHONPATH=$PYTHONPATH:$INSTALL_DIR;;
-  *) MODE=bdist_egg ;;
-esac
-
-$PYTHON setup_client.py $MODE --dist-dir=$INSTALL_DIR
-$PYTHON setup_common.py $MODE --dist-dir=$INSTALL_DIR
-$PYTHON setup_admin.py $MODE --dist-dir=$INSTALL_DIR
-$PYTHON setup_ctl.py $MODE --dist-dir=$INSTALL_DIR
-$PYTHON setup_tests.py $MODE --dist-dir=$INSTALL_DIR
-$PYTHON setup_storage.py $MODE --dist-dir=$INSTALL_DIR
-$PYTHON setup_master.py $MODE --dist-dir=$INSTALL_DIR
-

setup.py

-from setuptools import setup, find_packages
-
-setup(name='neo',
-    version='0.1.0',
-    description='Distributed, redundant and transactional storage for ZODB',
-    author='NEOPPOD',
-    author_email='neo-dev@erp5.org',
-    url='http://www.neoppod.org/',
-    license="GPL 2",
-    packages=find_packages(),
-    package_dir={'neo': 'neo'},
-    install_requires=[
-        'ZODB3',
-    ],
-    extras_require={
-        'storage': ['MySQL-python'],
-        'test': ['MySQL-python', 'mock'],
-    },
-    entry_points={
+# WKRD: Official Python packaging and buildout are not designed to build
+#       several binary packages from a single source one. This setup.py
+#       contains several hacks to work around this horrible limitation, and
+#       make it possible to develop several eggs in the same tree/repository:
+#       - in development mode, it behaves somehow like a mono-package
+#       - otherwise, sdist/bdist commands produces several packages
+
+description = 'Distributed, redundant and transactional storage for ZODB'
+
+setup_common = dict(
+    version = '1.0a1',
+    author = 'NEOPPOD',
+    author_email = 'neo-dev@erp5.org',
+    url = 'http://www.neoppod.org/',
+    license = 'GPL 2+',
+    install_requires = ['neoppod'],
+    namespace_packages = ['neo', 'neo.scripts'],
+    zip_safe = True,
+)
+
+import os, sys
+from distutils.filelist import findall
+
+try:
+    setup_only, = [l[5:].strip() for l in open('PKG-INFO') if l[:5] == 'Name:']
+except IOError:
+    setup_only = None
+
+def setup(name, packages=(), extra_files=(), _done=[], **kw):
+    if setup_only and setup_only != name or \
+       _done and ('--help' in sys.argv or '--help-commands' in sys.argv):
+        return
+    from distutils.command.build import build
+    from setuptools import find_packages, setup
+    from setuptools.command import egg_info, develop
+    kw['packages'] = sum((isinstance(p, basestring) and [p] or
+        list(p[:1]) + [p[0] + '.' + x for x in find_packages(*p[1:])]
+        for p in packages), [])
+    # monkey-patch to build package in different folders
+    build_initialize_options = build.initialize_options
+    def initialize_options(self):
+        build_initialize_options(self)
+        self.build_base = os.path.join(self.build_base, name)
+    # per-package manifest, instead of walking files under revision control
+    walk_revctrl = egg_info.walk_revctrl
+    # create only 1 egg-link for buildout
+    develop_finalize_options = develop.develop.finalize_options
+    def finalize_options(self):
+        develop_finalize_options(self)
+        self.egg_link = os.devnull
+    try:
+        build.initialize_options = initialize_options
+        egg_info.walk_revctrl = lambda *args, **kw: extra_files
+        if _done:
+            develop.develop.finalize_options = finalize_options
+        setup(name = name, **dict(setup_common, **kw))
+    finally:
+        develop.develop.finalize_options = develop_finalize_options
+        build.initialize_options = build_initialize_options
+        egg_info.walk_revctrl = walk_revctrl
+    _done.append(name)
+
+setup(
+    name = 'neoppod',
+    description = description + ' - Common part',
+    packages = ['neo', ('neo.lib', 'neo/lib')],
+    # Raah!!! I wish I could write something like (for 'neoppod'):
+    #  install_requires = ['python>=2.5|ctypes']
+    install_requires = (),
+    namespace_packages = ['neo'],
+    extra_files = ['TODO'],
+)
+
+setup(
+    name = 'neoadmin',
+    description = description + ' - Admin part',
+    packages = ['neo', ('neo.admin', 'neo/admin')],
+    py_modules = ['neo.scripts.neoadmin'],
+    entry_points = {
         'console_scripts': [
             'neoadmin=neo.scripts.neoadmin:main',
-            'neoctl=neo.scripts.neoctl:main',
-            'neomaster=neo.scripts.neomaster:main',
-            'neomigrate=neo.scripts.neomigrate:main',
-            'neostorage=neo.scripts.neostorage:main',
+        ],
+    },
+    extra_files = ['neo.conf'],
+)
+
+setup(
+    name = 'neoclient',
+    description = description + ' - Client part',
+    packages = ['neo', ('neo.client', 'neo/client')],
+    install_requires = ['neoppod', 'ZODB3 >= 3.9'],
+    py_modules = ['neo.scripts.neomigrate'],
+    entry_points = {
+        'console_scripts': [
+            'neoadmin=neo.scripts.neomigrate:main',
         ],
     },
     package_data = {
-        'neo': [
+        'neo.client': [
             'component.xml',
         ],
     },
     zip_safe=False,
 )
 
+setup(
+    name = 'neoctl',
+    description = description + ' - Controller part',
+    packages = ['neo', ('neo.neoctl', 'neo/neoctl')],
+    py_modules = ['neo.scripts.neomigrate'],
+    entry_points = {
+        'console_scripts': [
+            'neoctl=neo.scripts.neomigrate:main',
+        ],
+    },
+)
+
+setup(
+    name = 'neomaster',
+    description = description + ' - Master part',
+    packages = ['neo', ('neo.master', 'neo/master')],
+    py_modules = ['neo.scripts.neomaster'],
+    entry_points = {
+        'console_scripts': [
+            'neomaster=neo.scripts.neomaster:main',
+        ],
+    },
+    extra_files = ['neo.conf'],
+)
+
+setup(
+    name = 'neostorage',
+    description = description + ' - Storage part',
+    packages = ['neo', ('neo.storage', 'neo/storage')],
+    py_modules = ['neo.scripts.neostorage'],
+    entry_points = {
+        'console_scripts': [
+            'neostorage=neo.scripts.neostorage:main',
+        ],
+    },
+    extras_require = {
+        'btree': ['ZODB3'],
+        'mysqldb': ['MySQL-python'],
+    },
+    extra_files = ['neo.conf'],
+)
+
+if setup_only in ('neotests', None) and not os.path.exists('mock.py'):
+    import cStringIO, md5, urllib, zipfile
+    mock_py = zipfile.ZipFile(cStringIO.StringIO(urllib.urlopen(
+      'http://downloads.sf.net/sourceforge/python-mock/pythonmock-0.1.0.zip'
+      ).read())).read('mock.py')
+    if md5.md5(mock_py).hexdigest() != '79f42f390678e5195d9ce4ae43bd18ec':
+        raise EnvironmentError("MD5 checksum mismatch downloading 'mock.py'")
+    open('mock.py', 'w').write(mock_py)
+
+setup(
+    name = 'neotests',
+    description = description + ' - Testing part',
+    packages = ['neo', ('neo.tests', 'neo/tests')],
+    install_requires = [
+        'neoadmin',
+        'neoclient',
+        'neoctl',
+        'neomaster',
+        'neostorage[btree, mysqldb]',
+        'psutil',
+        'zope.testing',
+    ],
+    py_modules = ['mock', 'neo.scripts.runner'],
+    entry_points = {
+        'console_scripts': [
+            'neotestrunner=neo.scripts.runner:main',
+        ],
+    },
+    extra_files = ['TESTS.txt'] + findall('tools'),
+)

setup_admin.py

-from setuptools import setup, find_packages
-
-setup(name='neoadmin',
-
-    version=
-      '0.1.0',
-    description=
-      'Distributed, redundant and transactional storage for ZODB-Admin part',
-    author=
-      'NEOPPOD',
-    author_email=
-      'neo-dev@erp5.org',
-    url=
-      'http://www.neoppod.org/',
-    license=
-      "GPL 2",
-
-    py_modules=[
-      'neo.scripts.neoadmin'
-      ],
-
-    packages=['neo.admin'],
-
-    package_dir={
-      'neo':'neo',
-    },
-
-    namespace_packages=['neo','neo.scripts'],
-
-    install_requires=[
-      'neo',
-    ],
-
-    entry_points = {
-        'console_scripts': [
-          'neoadmin=neo.scripts.neoadmin:main',
-        ],
-    },
-    zip_safe=False,
-)
-

setup_client.py

-from setuptools import setup, find_packages
-
-setup(name='neoclient',
-
-    version=
-      '0.1.0',
-    description=
-      'Distributed, redundant and transactional storage for ZODB-Admin part',
-    author=
-      'NEOPPOD',
-    author_email=
-      'neo-dev@erp5.org',
-    url=
-      'http://www.neoppod.org/',
-    license=
-      "GPL 2",
-
-    py_modules=[
-     'neo.scripts.neomigrate',
-      ],
-
-    packages=['neo.client','neo.client.handlers'],
-
-    package_dir={
-      'neo':'neo',
-    },
-
-    namespace_packages=['neo','neo.client'],
-
-    install_requires=[
-      'neo',
-      'ZODB3',
-    ],
-
-    zip_safe=False,
-)
-

setup_common.py

-from setuptools import setup, find_packages
-
-setup(name='neo',
-    version='0.1.0',
-    description='Distributed, redundant and transactional storage for ZODB- Common part',
-    author='NEOPPOD',
-    author_email='neo-dev@erp5.org',
-    url='http://www.neoppod.org/',
-    license="GPL 2",
-
-    packages=['neo.lib'],
-    package_dir={
-            'neo':'neo',
-    },
-    namespace_packages=['neo'],
-
-    package_data = {
-        'neo': [
-            'component.xml',
-        ],
-    },
-    zip_safe=False,
-)
-

setup_ctl.py

-from setuptools import setup, find_packages
-
-setup(name='neoctl',
-
-    version=
-      '0.1.0',
-    description=
-      'Distributed, redundant and transactional storage for ZODB-Admin part',
-    author=
-      'NEOPPOD',
-    author_email=
-      'neo-dev@erp5.org',
-    url=
-      'http://www.neoppod.org/',
-    license=
-      "GPL 2",
-
-    py_modules=[
-      'neo.scripts.neoctl'
-      ],
-
-    packages=['neo.neoctl'],
-
-    package_dir={
-      'neo':'neo',
-    },
-
-    namespace_packages=['neo','neo.scripts'],
-
-    install_requires=[
-      'neo',
-    ],
-
-    entry_points = {
-        'console_scripts': [
-          'neoctl=neo.scripts.neoctl:main',
-        ],
-    },
-    zip_safe=False,
-)
-

setup_master.py

-from setuptools import setup, find_packages
-
-setup(name='neomaster',
-
-    version=
-      '0.1.0',
-    description=
-      'Distributed, redundant and transactional storage for ZODB-Admin part',
-    author=
-      'NEOPPOD',
-    author_email=
-      'neo-dev@erp5.org',
-    url=
-      'http://www.neoppod.org/',
-    license=
-      "GPL 2",
-
-    py_modules=[
-      'neo.scripts.neomaster',
-      ],
-
-    packages=['neo.master','neo.master.handlers'],
-
-    package_dir={
-      'neo':'neo',
-    },
-
-    namespace_packages=['neo','neo.scripts'],
-
-    install_requires=[
-      'neo',
-    ],
-
-    entry_points = {
-        'console_scripts': [
-          'neomaster=neo.scripts.neomaster:main',
-        ],
-    },
-    zip_safe=False,
-)
-

setup_storage.py

-from setuptools import setup, find_packages
-
-setup(name='neostorage',
-
-    version=
-      '0.1.0',
-    description=
-      'Distributed, redundant and transactional storage for ZODB-Admin part',
-    author=
-      'NEOPPOD',
-    author_email=
-      'neo-dev@erp5.org',
-    url=
-      'http://www.neoppod.org/',
-    license=
-      "GPL 2",
-
-    py_modules=[
-      'neo.protocol',
-      'neo.scripts.neostorage'
-      ],
-
-    packages=['neo.storage','neo.storage.database','neo.storage.handlers'],
-
-    package_dir={
-      'neo':'neo',
-    },
-
-    namespace_packages=['neo','neo.scripts'],
-
-    install_requires=[
-      'neo',
-      'MySQL-python',
-    ],
-
-    entry_points = {
-        'console_scripts': [
-          'neostorage=neo.scripts.neostorage:main',
-        ],
-    },
-    zip_safe=False,
-)
-

setup_tests.py

-from setuptools import setup, find_packages
-
-setup(name='neotests',
-
-    version=
-      '0.1.0',
-    description=
-      'Distributed, redundant and transactional storage for ZODB-Admin part',
-    author=
-      'NEOPPOD',
-    author_email=
-      'neo-dev@erp5.org',
-    url=
-      'http://www.neoppod.org/',
-    license=
-      "GPL 2",
-
-    py_modules=[
-      'neo.scripts.runner'
-      ],
-
-    packages=['neo.tests','neo.tests.functional','neo.tests.master','neo.tests.storage','neo.tests.zodb','neo.tests.client'],
-
-    package_dir={
-      'neo':'neo',
-    },
-
-    namespace_packages=['neo','neo.scripts'],
-
-    install_requires=[
-      'neo',
-      'neoadmin',
-      'neostorage',
-      'neoclient',
-      'neomaster',
-      'neoctl',
-      'mock'
-    ],
-
-    entry_points = {
-        'console_scripts': [
-                'neotestrunner=neo.scripts.runner:main',
-        ],
-    },
-    zip_safe=False,
-)
-