Skip to content

Z
Zope
Commit 87c90136 authored by Hanno Schlichting's avatar Hanno Schlichting
Browse files
Options

Factored out Products.BTreeFolder2

parent 324c96f5
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 8 additions and 1178 deletions
buildout.cfg
View file @ 87c90136
......@@ -49,6 +49,7 @@ eggs =
Missing
MultiMapping
Persistence
Products.BTreeFolder2
Products.ExternalMethod
Products.PythonScripts
Products.ZCTextIndex
......
doc/CHANGES.rst
View file @ 87c90136
......@@ -36,10 +36,10 @@ Restructuring
- Avoid using the ``Products.PythonScripts.standard`` module inside the
database manager ZMI.
- Factored out the `Products.ExternalMethod`, `Products.MIMETools`,
`Products.OFSP` and `Products.PythonScripts` packages into their own
distributions. They will no longer be included by default in Zope 2.14 but
live on as independent add-ons.
- Factored out the `Products.BTreeFolder2`, `Products.ExternalMethod`,
`Products.MIMETools`, `Products.OFSP` and `Products.PythonScripts` packages
into their own distributions. They will no longer be included by default in
Zope 2.14 but live on as independent add-ons.
- Factored out the `Products.ZSQLMethods` into its own distribution. The
distribution also includes the `Shared.DC.ZRDB` code. The Zope2 distribution
......
setup.py
View file @ 87c90136
......@@ -98,6 +98,7 @@ setup(name='Zope2',
'zope.traversing',
'zope.viewlet',
# BBB optional dependencies to be removed in Zope 2.14
'Products.BTreeFolder2',
'Products.ExternalMethod',
'Products.MIMETools',
'Products.OFSP',
......
sources.cfg
View file @ 87c90136
......@@ -9,6 +9,7 @@ Missing = svn ^/Missing/trunk
MultiMapping = svn ^/MultiMapping/trunk
nt_svcutils = svn ^/nt_svcutils/trunk
Persistence = svn ^/Persistence/trunk
Products.BTreeFolder2 = svn ^/Products.BTreeFolder2/trunk
Products.ExternalMethod = svn ^/Products.ExternalMethod/trunk
Products.MIMETools = svn ^/Products.MIMETools/trunk
Products.OFSP = svn ^/Products.OFSP/trunk
......
src/Products/BTreeFolder2/BTreeFolder2.py deleted 100644 → 0
View file @ 324c96f5
This diff is collapsed.
src/Products/BTreeFolder2/CHANGES.txt deleted 100644 → 0
View file @ 324c96f5
Version 1.0.2
- Made CMFBTreeFolder compatible with CMF 1.5+
Version 1.0.1
- ConflictError was swallowed by _delObject. This could break code
expecting to do cleanups before deletion.
- Renamed hasObject() to has_key(). hasObject() conflicted with
another product.
- You can now visit objects whose names have a trailing space.
Version 1.0
- BTreeFolder2s now use an icon contributed by Chris Withers.
- Since recent ZODB releases have caused minor corruption in BTrees,
there is now a manage_cleanup method for fixing damaged BTrees
contained in BTreeFolders.
Version 0.5.1
- Fixed the CMFBTreeFolder constructor.
src/Products/BTreeFolder2/README.txt deleted 100644 → 0
View file @ 324c96f5
Contact
=======
Shane Hathaway
Zope Corporation
shane at zope dot com
BTreeFolder2 Product
====================
BTreeFolder2 is a Zope product that acts like a Zope folder but can
store many more items.
When you fill a Zope folder with too many items, both Zope and your
browser get overwhelmed. Zope has to load and store a large folder
object, and the browser has to render large HTML tables repeatedly.
Zope can store a lot of objects, but it has trouble storing a lot of
objects in a single standard folder.
Zope Corporation once had an extensive discussion on the subject. It
was decided that we would expand standard folders to handle large
numbers of objects gracefully. Unfortunately, Zope folders are used
and extended in so many ways today that it would be difficult to
modify standard folders in a way that would be compatible with all
Zope products.
So the BTreeFolder product was born. It stored all subobjects in a
ZODB BTree, a structure designed to allow many items without loading
them all into memory. It also rendered the contents of the folder as
a simple select list rather than a table. Most browsers have no
trouble rendering large select lists.
But there was still one issue remaining. BTreeFolders still stored
the ID of all subobjects in a single database record. If you put tens
of thousands of items in a single BTreeFolder, you would still be
loading and storing a multi-megabyte folder object. Zope can do this,
but not quickly, and not without bloating the database.
BTreeFolder2 solves this issue. It stores not only the subobjects but
also the IDs of the subobjects in a BTree. It also batches the list
of items in the UI, showing only 1000 items at a time. So if you
write your application carefully, you can use a BTreeFolder2 to store
as many items as will fit in physical storage.
There are products that depend on the internal structure of the
original BTreeFolder, however. So rather than risk breaking those
products, the product has been renamed. You can have both products
installed at the same time. If you're developing new applications,
you should use BTreeFolder2.
Installation
============
Untar BTreeFolder2 in your Products directory and restart Zope.
BTreeFolder2 will now be available in your "Add" drop-down.
Additionally, if you have CMF installed, the BTreeFolder2 product also
provides the "CMF BTree Folder" addable type.
Usage
=====
The BTreeFolder2 user interface shows a list of items rather than a
series of checkboxes. To visit an item, select it in the list and
click the "edit" button.
BTreeFolder2 objects provide Python dictionary-like methods to make them
easier to use in Python code than standard folders::
has_key(key)
keys()
values()
items()
get(key, default=None)
__len__()
keys(), values(), and items() return sequences, but not necessarily
tuples or lists. Use len(folder) to call the __len__() method. The
objects returned by values() and items() have acquisition wrappers.
BTreeFolder2 also provides a method for generating unique,
non-overlapping IDs::
generateId(prefix='item', suffix='', rand_ceiling=999999999)
The ID returned by this method is guaranteed to not clash with any
other ID in the folder. Use the returned value as the ID for new
objects. The generated IDs tend to be sequential so that objects that
are likely related in some way get loaded together.
BTreeFolder2 implements the full Folder interface, with the exception
that the superValues() method does not return any items. To implement
the method in the way the Zope codebase expects would undermine the
performance benefits gained by using BTreeFolder2.
Repairing BTree Damage
======================
Certain ZODB bugs in the past have caused minor corruption in BTrees.
Fortunately, the damage is apparently easy to repair. As of version
1.0, BTreeFolder2 provides a 'manage_cleanup' method that will check
the internal structure of existing BTreeFolder2 instances and repair
them if necessary. Many thanks to Tim Peters, who fixed the BTrees
code and provided a function for checking a BTree.
Visit a BTreeFolder2 instance through the web as a manager. Add
"manage_cleanup" to the end of the URL and request that URL. It may
take some time to load and fix the entire structure. If problems are
detected, information will be added to the event log.
src/Products/BTreeFolder2/__init__.py deleted 100644 → 0
View file @ 324c96f5
##############################################################################
#
# Copyright (c) 2001, 2002 Zope Foundation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE
#
##############################################################################
__doc__='''BTreeFolder2 Product Initialization
$Id: __init__.py,v 1.4 2003/08/21 17:03:52 shane Exp $'''
__version__='$Revision: 1.4 $'[11:-2]
import BTreeFolder2
def initialize(context):
context.registerClass(
BTreeFolder2.BTreeFolder2,
constructors=(BTreeFolder2.manage_addBTreeFolderForm,
BTreeFolder2.manage_addBTreeFolder),
icon='btreefolder2.gif',
)
src/Products/BTreeFolder2/contents.dtml deleted 100644 → 0
View file @ 324c96f5
<dtml-let form_title="'Contents'">
<dtml-if manage_page_header>
<dtml-var manage_page_header>
<dtml-else>
<html><head><title>&dtml-form_title;</title></head>
<body bgcolor="#ffffff">
</dtml-if>
</dtml-let>
<dtml-var manage_tabs>
<script type="text/javascript">
<!--
isSelected = false;
function toggleSelect() {
elem = document.objectItems.elements['ids:list'];
if (isSelected == false) {
for (i = 0; i < elem.options.length; i++) {
elem.options[i].selected = true;
}
isSelected = true;
document.objectItems.selectButton.value = "Deselect All";
return isSelected;
}
else {
for (i = 0; i < elem.options.length; i++) {
elem.options[i].selected = false;
}
isSelected = false;
document.objectItems.selectButton.value = "Select All";
return isSelected;
}
}
//-->
</script>
<dtml-unless skey><dtml-call expr="REQUEST.set('skey', 'id')"></dtml-unless>
<dtml-unless rkey><dtml-call expr="REQUEST.set('rkey', '')"></dtml-unless>
<!-- Add object widget -->
<br />
<dtml-if filtered_meta_types>
<table width="100%" cellspacing="0" cellpadding="0" border="0">
<tr>
<td align="left" valign="top">&nbsp;</td>
<td align="right" valign="top">
<div class="form-element">
<form action="&dtml-URL1;/" method="get">
<dtml-if "_.len(filtered_meta_types) > 1">
<select class="form-element" name=":action"
onChange="location.href='&dtml-URL1;/'+this.options[this.selectedIndex].value">
<option value="manage_workspace" disabled>Select type to add...</option>
<dtml-in filtered_meta_types mapping sort=name>
<option value="&dtml.url_quote-action;">&dtml-name;</option>
</dtml-in>
</select>
<input class="form-element" type="submit" name="submit" value=" Add " />
<dtml-else>
<dtml-in filtered_meta_types mapping sort=name>
<input type="hidden" name=":method" value="&dtml.url_quote-action;" />
<input class="form-element" type="submit" name="submit" value=" Add &dtml-name;" />
</dtml-in>
</dtml-if>
</form>
</div>
</td>
</tr>
</table>
</dtml-if>
<form action="&dtml-URL1;/" name="objectItems" method="post">
<dtml-if objectCount>
<dtml-with expr="getBatchObjectListing(REQUEST)" mapping>
<p>
<dtml-if prev_batch_url><a href="&dtml-prev_batch_url;">&lt;&lt;</a></dtml-if>
<em>Items <dtml-var b_start> through <dtml-var b_end> of <dtml-var objectCount></em>
<dtml-if next_batch_url><a href="&dtml-next_batch_url;">&gt;&gt;</a></dtml-if>
</p>
<dtml-var formatted_list>
<table cellspacing="0" cellpadding="2" border="0">
<tr>
<td align="left" valign="top" width="16"></td>
<td align="left" valign="top">
<div class="form-element">
<input class="form-element" type="submit"
name="manage_object_workspace:method" value="Edit" />
<dtml-unless dontAllowCopyAndPaste>
<input class="form-element" type="submit" name="manage_renameForm:method"
value="Rename" />
<input class="form-element" type="submit" name="manage_cutObjects:method"
value="Cut" />
<input class="form-element" type="submit" name="manage_copyObjects:method"
value="Copy" />
<dtml-if cb_dataValid>
<input class="form-element" type="submit" name="manage_pasteObjects:method"
value="Paste" />
</dtml-if>
</dtml-unless>
<dtml-if "_.SecurityCheckPermission('Delete objects',this())">
<input class="form-element" type="submit" name="manage_delObjects:method"
value="Delete" />
</dtml-if>
<dtml-if "_.SecurityCheckPermission('Import/Export objects', this())">
<input class="form-element" type="submit"
name="manage_importExportForm:method"
value="Import/Export" />
</dtml-if>
<script type="text/javascript">
<!--
if (document.forms[0]) {
document.write('<input class="form-element" type="submit" name="selectButton" value="Select All" onClick="toggleSelect(); return false">')
}
//-->
</script>
</div>
</td>
</tr>
</table>
</dtml-with>
<dtml-else>
<table cellspacing="0" cellpadding="2" border="0">
<tr>
<td>
<div class="std-text">
There are currently no items in <em>&dtml-title_or_id;</em>
<br /><br />
</div>
<dtml-unless dontAllowCopyAndPaste>
<dtml-if cb_dataValid>
<div class="form-element">
<input class="form-element" type="submit" name="manage_pasteObjects:method"
value="Paste" />
</div>
</dtml-if>
</dtml-unless>
<dtml-if "_.SecurityCheckPermission('Import/Export objects', this())">
<input class="form-element" type="submit"
name="manage_importExportForm:method" value="Import/Export" />
</dtml-if>
</td>
</tr>
</table>
</dtml-if>
</form>
<dtml-if update_menu>
<script type="text/javascript">
<!--
window.parent.update_menu();
//-->
</script>
</dtml-if>
<dtml-if manage_page_footer>
<dtml-var manage_page_footer>
<dtml-else>
</body></html>
</dtml-if>
src/Products/BTreeFolder2/folderAdd.dtml deleted 100644 → 0
View file @ 324c96f5
<dtml-let form_title="'Add BTreeFolder2'">
<dtml-if manage_page_header>
<dtml-var manage_page_header>
<dtml-var manage_form_title>
<dtml-else>
<html><head><title>&dtml-form_title;</title></head>
<body bgcolor="#ffffff">
<h2>&dtml-form_title;</h2>
</dtml-if>
</dtml-let>
<p class="form-help">
A Folder contains other objects. Use Folders to organize your
web objects in to logical groups.
</p>
<p class="form-help">
A BTreeFolder2 may be able to handle a larger number
of objects than a standard folder because it does not need to
activate other subobjects in order to access a single subobject.
It is more efficient than the original BTreeFolder product,
but does not provide attribute access.
</p>
<FORM ACTION="manage_addBTreeFolder" METHOD="POST">
<table cellspacing="0" cellpadding="2" border="0">
<tr>
<td align="left" valign="top">
<div class="form-label">
Id
</div>
</td>
<td align="left" valign="top">
<input type="text" name="id" size="40" />
</td>
</tr>
<tr>
<td align="left" valign="top">
<div class="form-optional">
Title
</div>
</td>
<td align="left" valign="top">
<input type="text" name="title" size="40" />
</td>
</tr>
<tr>
<td align="left" valign="top">
</td>
<td align="left" valign="top">
<div class="form-element">
<input class="form-element" type="submit" name="submit"
value="Add" />
</div>
</td>
</tr>
</table>
</form>
<dtml-if manage_page_footer>
<dtml-var manage_page_footer>
<dtml-else>
</body></html>
</dtml-if>
src/Products/BTreeFolder2/tests/testBTreeFolder2.py deleted 100644 → 0
View file @ 324c96f5
##############################################################################
#
# Copyright (c) 2001, 2002 Zope Foundation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE
#
##############################################################################
"""Unit tests for BTreeFolder2.
$Id: testBTreeFolder2.py,v 1.8 2004/03/15 20:31:40 shane Exp $
"""
import unittest
import ZODB
import Testing
import Zope2
from Products.BTreeFolder2.BTreeFolder2 \
import BTreeFolder2, ExhaustedUniqueIdsError
from OFS.ObjectManager import BadRequestException
from OFS.Folder import Folder
from Acquisition import aq_base
class BTreeFolder2Tests(unittest.TestCase):
def getBase(self, ob):
# This is overridden in subclasses.
return aq_base(ob)
def setUp(self):
self.f = BTreeFolder2('root')
ff = BTreeFolder2('item')
self.f._setOb(ff.id, ff)
self.ff = self.f._getOb(ff.id)
def testAdded(self):
self.assertEqual(self.ff.id, 'item')
def testCount(self):
self.assertEqual(self.f.objectCount(), 1)
self.assertEqual(self.ff.objectCount(), 0)
self.assertEqual(len(self.f), 1)
self.assertEqual(len(self.ff), 0)
def testObjectIds(self):
self.assertEqual(list(self.f.objectIds()), ['item'])
self.assertEqual(list(self.f.keys()), ['item'])
self.assertEqual(list(self.ff.objectIds()), [])
f3 = BTreeFolder2('item3')
self.f._setOb(f3.id, f3)
lst = list(self.f.objectIds())
lst.sort()
self.assertEqual(lst, ['item', 'item3'])
def testObjectIdsWithMetaType(self):
f2 = Folder()
f2.id = 'subfolder'
self.f._setOb(f2.id, f2)
mt1 = self.ff.meta_type
mt2 = Folder.meta_type
self.assertEqual(list(self.f.objectIds(mt1)), ['item'])
self.assertEqual(list(self.f.objectIds((mt1,))), ['item'])
self.assertEqual(list(self.f.objectIds(mt2)), ['subfolder'])
lst = list(self.f.objectIds([mt1, mt2]))
lst.sort()
self.assertEqual(lst, ['item', 'subfolder'])
self.assertEqual(list(self.f.objectIds('blah')), [])
def testObjectValues(self):
values = self.f.objectValues()
self.assertEqual(len(values), 1)
self.assertEqual(values[0].id, 'item')
# Make sure the object is wrapped.
self.assert_(values[0] is not self.getBase(values[0]))
def testObjectItems(self):
items = self.f.objectItems()
self.assertEqual(len(items), 1)
id, val = items[0]
self.assertEqual(id, 'item')
self.assertEqual(val.id, 'item')
# Make sure the object is wrapped.
self.assert_(val is not self.getBase(val))
def testHasKey(self):
self.assert_(self.f.hasObject('item')) # Old spelling
self.assert_(self.f.has_key('item')) # New spelling
def testDelete(self):
self.f._delOb('item')
self.assertEqual(list(self.f.objectIds()), [])
self.assertEqual(self.f.objectCount(), 0)
def testObjectMap(self):
map = self.f.objectMap()
self.assertEqual(list(map), [{'id': 'item', 'meta_type':
self.ff.meta_type}])
# I'm not sure why objectMap_d() exists, since it appears to be
# the same as objectMap(), but it's implemented by Folder.
self.assertEqual(list(self.f.objectMap_d()), list(self.f.objectMap()))
def testObjectIds_d(self):
self.assertEqual(self.f.objectIds_d(), {'item': 1})
def testCheckId(self):
self.assertEqual(self.f._checkId('xyz'), None)
self.assertRaises(BadRequestException, self.f._checkId, 'item')
self.assertRaises(BadRequestException, self.f._checkId, 'REQUEST')
def testSetObject(self):
f2 = BTreeFolder2('item2')
self.f._setObject(f2.id, f2)
self.assert_(self.f.has_key('item2'))
self.assertEqual(self.f.objectCount(), 2)
def testWrapped(self):
# Verify that the folder returns wrapped versions of objects.
base = self.getBase(self.f._getOb('item'))
self.assert_(self.f._getOb('item') is not base)
self.assert_(self.f['item'] is not base)
self.assert_(self.f.get('item') is not base)
self.assert_(self.getBase(self.f._getOb('item')) is base)
def testGenerateId(self):
ids = {}
for n in range(10):
ids[self.f.generateId()] = 1
self.assertEqual(len(ids), 10) # All unique
for id in ids.keys():
self.f._checkId(id) # Must all be valid
def testGenerateIdDenialOfServicePrevention(self):
for n in range(10):
item = Folder()
item.id = 'item%d' % n
self.f._setOb(item.id, item)
self.f.generateId('item', rand_ceiling=20) # Shouldn't be a problem
self.assertRaises(ExhaustedUniqueIdsError,
self.f.generateId, 'item', rand_ceiling=9)
def testReplace(self):
old_f = Folder()
old_f.id = 'item'
inner_f = BTreeFolder2('inner')
old_f._setObject(inner_f.id, inner_f)
self.ff._populateFromFolder(old_f)
self.assertEqual(self.ff.objectCount(), 1)
self.assert_(self.ff.has_key('inner'))
self.assertEqual(self.getBase(self.ff._getOb('inner')), inner_f)
def testObjectListing(self):
f2 = BTreeFolder2('somefolder')
self.f._setObject(f2.id, f2)
# Hack in an absolute_url() method that works without context.
self.f.absolute_url = lambda: ''
info = self.f.getBatchObjectListing()
self.assertEqual(info['b_start'], 1)
self.assertEqual(info['b_end'], 2)
self.assertEqual(info['prev_batch_url'], '')
self.assertEqual(info['next_batch_url'], '')
self.assert_(info['formatted_list'].find('</select>') > 0)
self.assert_(info['formatted_list'].find('item') > 0)
self.assert_(info['formatted_list'].find('somefolder') > 0)
# Ensure batching is working.
info = self.f.getBatchObjectListing({'b_count': 1})
self.assertEqual(info['b_start'], 1)
self.assertEqual(info['b_end'], 1)
self.assertEqual(info['prev_batch_url'], '')
self.assert_(info['next_batch_url'] != '')
self.assert_(info['formatted_list'].find('item') > 0)
self.assert_(info['formatted_list'].find('somefolder') < 0)
info = self.f.getBatchObjectListing({'b_start': 2})
self.assertEqual(info['b_start'], 2)
self.assertEqual(info['b_end'], 2)
self.assert_(info['prev_batch_url'] != '')
self.assertEqual(info['next_batch_url'], '')
self.assert_(info['formatted_list'].find('item') < 0)
self.assert_(info['formatted_list'].find('somefolder') > 0)
def testObjectListingWithSpaces(self):
# The option list must use value attributes to preserve spaces.
name = " some folder "
f2 = BTreeFolder2(name)
self.f._setObject(f2.id, f2)
self.f.absolute_url = lambda: ''
info = self.f.getBatchObjectListing()
expect = '<option value="%s">%s</option>' % (name, name)
self.assert_(info['formatted_list'].find(expect) > 0)
def testCleanup(self):
self.assert_(self.f._cleanup())
key = TrojanKey('a')
self.f._tree[key] = 'b'
self.assert_(self.f._cleanup())
key.value = 'z'
# With a key in the wrong place, there should now be damage.
self.assert_(not self.f._cleanup())
# Now it's fixed.
self.assert_(self.f._cleanup())
# Verify the management interface also works,
# but don't test return values.
self.f.manage_cleanup()
key.value = 'a'
self.f.manage_cleanup()
class TrojanKey:
"""Pretends to be a consistent, immutable, humble citizen...
then sweeps the rug out from under the BTree.
"""
def __init__(self, value):
self.value = value
def __cmp__(self, other):
return cmp(self.value, other)
def __hash__(self):
return hash(self.value)
def test_suite():
return unittest.TestSuite((
unittest.makeSuite(BTreeFolder2Tests),
))
if __name__ == '__main__':
unittest.main(defaultTest='test_suite')
versions.cfg
View file @ 87c90136
......@@ -15,6 +15,7 @@ Missing = 2.13.1
MultiMapping = 2.13.0
nt-svcutils = 2.13.0
Persistence = 2.13.2
Products.BTreeFolder2 = 2.13.0
Products.ExternalMethod = 2.13.0
Products.MIMETools = 2.13.0
Products.OFSP = 2.13.1
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7