Add __getstate__() and __setstate__() methods.

Thanks to Chris McDonough for helping me see the problem.

There are extensive comments that discuss the backwards compatibility
issues.  Basically, we must be careful that the pickles these methods
operate on are just like the pickles generated by the old code, or
developers couldn't switch back and forth between new and old code.
That's a incompatibility we aren't willing to live with for now --
perhaps not ever.

    # If the internal representation of PersistentMapping changes,
    # it causes compatibility problems for pickles generated by
    # different versions of the code.  Compatibility works in both
    # directions, because an application may want to share a database
    # between applications using different versions of the code.

    # Effectively, the original rep is part of the "API."  To provide
    # full compatibility, the getstate and setstate must read and
    # right objects using the old rep.

    # As a result, the PersistentMapping must save and restore the
    # actual internal dictionary using the name _container.

As the checkin message on the branch said, this should make the new
code work correctly using pickles generated by the old code.

The doc string for __changed__() says that it is deprecated, but
presumably the calls here were old and just never got updated.

The implementation now defers all implementation decisions to UserDict
and merely handles updates to _p_changed in the PersistentMapping
implementation.  This change simplifies the implementation and ensures
that it implements all the new methods of dictionaries like popitem()
and setdefault().

There is one (important?) behavioral change:  The old implementation
of the keys() method did not return keys that were strings starting
with underscore.  This behavior was undocumented and could lead to
problems for implementations that expect keys() to working like a
regular mapping.

    The feature was removed after verifying that the Zope test suite
    does not depend on this behavior.

An internal implementation change is that the PersistentMapping does
not keep a copy of the keys() of the dict.  It simplifies the
implementation a lot to remove the cacheing and the benefit is
unclear.  It only benefits applications that call keys() frequently
without modifying the dict.  And it requires that two copies of the
list exist for each call to keys() which may require a lot of space.

Replaced it with a standard ValueError exception

to decref the revision string, added to the module dict with the keys
"__version__".  This fixes a small leaks detected by Insure.

Also, at the end of the function, we do a PyErr_Occurred() check and
throw a fatal error if true.  This makes this module's init function
more in line with other init functions in this package.

decref the revision string, added to the module dict with the keys
"__version__".  This fixes a small leaks detected by Insure.

Also, at the end of the function, we do a PyErr_Occurred() check and
throw a fatal error if true.  This makes this module's init function
more in line with other init functions in this package.

decref the revision string, added to the module dict with the keys
"__version__".  Do the same with the PyCObject set to the dict's
"CAPI" key.  This fixes small leaks detected by Insure.

Also, clean up the way the global TimeStamp static is handled on
creation failure.  We need to decref `s' whether or not TimeStamp was
properly created or not.  Also, at the end of the function, we do a
PyErr_Occurred() check and throw a fatal error if true.  This makes
this module's init function more in line with other init functions in
this package.

decref the revision string, added to the module dict with the keys
"__version__".  This fixes a small leaks detected by Insure.

Also, instead of returning from the init should the
PyString_FromString() of "TimeStamp.error fail, we simply take
precautions by XDECREF'ing it.  Then at the end of the function, we do
a PyErr_Occurred() check and throw a fatal error if true.  This makes
this module's init function more in line with other init functions in
this package.

module dictionary under the "__version__" key.  Plugs a small memory
leak caught by Insure.

The previous checkin stopped using the per-connection trigger, but
didn't stop creating it.  (Dang.)

Thanks to Shane for noticing.

based on:

  - transaction iteration/copy

  - scanning for transactions after bad data

This should allow recovery of data when:

  - only data records are damaged and when

  - multiple parts of a file are damaged

The interface has changed to not modify in place.

Other features:

  - Progress indicator

  - Verbose output

  - optional packing

  - index creation

DB.py: typo in comment
cPickleCache.c: fix cache sweep logic
cPersistence.c: small performance changes

Not sure how this one got missed.

The trunk now has the same code ZEO 1.0b5 plus a few minor changes.

acceptability of serials=None for the object-creation store() call.
Note that _dostore() itself never exercises this condition because it
coerces its own serial argument to ZERO (a.k.a. '\0'*8) when None is
passed in.

The Mapping_update() method iterated over a sequence of 2-tuples, but
held an extra reference to the sequence.  The sequence object was
DECREFed on an error exit but not on the normal return.

The fix reworks the control flow of the function to make the reference
counting behavior a little clearer:

   - Test for call with no arguments and return immediately if so.

   - If the argument is a sequence, don't INCREF it.

   - If the argument passed isn't a sequence, call its items() method
     and store the result in the local variables seq and items.

   - On exit, do Py_XDECREF(items).  This will do a decref only if a
     the second case above applied.

Two other minor nits:

   - Make consistent use of spaces around = in assignments (as opposed
     to a fix of both)

   - Remove assignment from conditional expression.

pack() uses a coarse-grain timer to decide which revisions to look at
during a pack.  The tests depend on a pack() called after a commit to
see that committed transaction, but the implementation can't guarantee
that it will.

In particular, pack() converts its time argument to a TimeStamp by
truncating to the nearest second.  If a transaction T1 commits and pack
is called (with the current time as its argument) during the same second,
the pack() will not consider T1.  This was causing intermittent, but
frequent test failures on Windows.

The fix is to add a small delay in the pack test cases to guarantee that
pack(time.time()) will always see the last committed transaction.

If the packpos returned by the read_index() call is 4, return immediately.
In this case, the very first transaction, which contains the root object,
is after the pack time.  This is a degenerate case, but it seems clearest
to do nothing.

This change eliminates an unexpected traceback in pack() that occurred
frequently, but not every time, when testing on Windows.  When
read_index() returned 4, a seek() call in _redundant_pack() would fail
with a bad argument.

Add an assertion to _redundant_pack() because it should never be called
with bogus arguments as a result of the other change.

Don't test for explicit end-of-line markers, since they vary across
platforms.  Instead, strip() the line read and test for the actual content.

If a storage doesn't support T.U. and is old enough, you need to first
check if it has a supportsTransactionalUndo attribute.

There are a few problems with Zope 2.3 that prevented the tests from
succeeding.  Since the 1.0 release must support Zope 2.3, we'll need
to hack the various tests to work around them.

    Assuming releases after 1.0 don't need to work with 2.3: After the
    1.0 final release, I'll tag both ZEO and ZODB trees and then undo
    all these silly changes.

Problems:

    - no support for transactional undo or getSerial()
    - FileStorage commitVersion() and abortVersion() didn't check for
      bogus arguments

This implementation is only faster for an object that is not modified
in a version.  For an object modified in a version, it should be about
the same as the default getSerial() in BaseStorage.

situation.

Didn't write a more interesting test in checkAbortVersion:

        # XXX Not sure I can write a test for getSerial() in the
        # presence of aborted versions, because FileStorage and
        # Berkeley storage give a different answer. I think Berkeley
        # is right and FS is wrong.

BaseStorage implements this in terms of modifiedInVersion() and
load().  Other storages may (perhaps even should) implement it more
efficiently.

now a full success :-)

(Minor change to tb handling to avoid unnecessary binding to None for
v and tb.)

Prevents memory leak for each _restore_index() call.