1. 26 Mar, 2018 1 commit
    • Jim Fulton's avatar
      - ZODB now uses pickle protocol 3 for both Python 2 and Python 3. (#194) · 12ee41c4
      Jim Fulton authored
      (Previously, protocol 2 was used for Python 2.)
      
        The zodbpickle package provides a `zodbpickle.binary` string type
        that should be used in Python 2 to cause binary strings to be saved
        in a pickle binary format, so they can be loaded correctly in
        Python 3.  Pickle protocol 3 is needed for this to work correctly.
      
      - Object identifiers in persistent references are saved as
        `zodbpickle.binary` strings in Python 2, so that they are loaded
        correctly in Python 3.
      12ee41c4
  2. 25 Mar, 2018 1 commit
  3. 10 Mar, 2018 1 commit
  4. 09 Mar, 2018 1 commit
  5. 15 Feb, 2018 1 commit
    • Jim Fulton's avatar
      Avoid seeking on import (#188) · 2115c90a
      Jim Fulton authored
      * Don't seek on imports
      
      (other than the possible seek for custom importers)
      
      We were seeking to handle blob markers. This has two major drawbacks:
      
      1. It wasn't possible to use a non-seekable file.  A use case for
         export/import is to copy database data.  An intermediate file, and
         associated I/O, could be avoided using a pipe, but pipes aren't
         seekable.
      
      2. Seeks cause file-buffer data to be discarded, making IO far more
         expensive.
      
      We didn't really need blob markers, because the preceeding blob data
      records serve as markers.  (Now we're stuck with them for backward
      compatibility.)
      
      * Make cp's buffer size larger and configurable.
      
      * Use the storage temprary directory when importing blobs
      
      To avoid an extra copy.
      
      Also, allow the copy (cp) buffer sie to be overridden on export.
      
      (I see no obvious way to plumb it on import. :( )
      
      * Oops, need to use a binary literal (Python 3)
      
      * Respond to PR comments
      2115c90a
  6. 14 Nov, 2017 1 commit
    • Jim Fulton's avatar
      Update the front page to be a little less negative. (#186) · de1f24ca
      Jim Fulton authored
      Especially with regard to write capacity and search.
      
      This hasn't been updated in a while and I feel that it is out of date.
      
      I feel that this could use more work, but I'm anxious to do an
      incremental improvement given the screencast I just released. :)
      de1f24ca
  7. 13 Nov, 2017 2 commits
  8. 12 Nov, 2017 1 commit
  9. 08 Nov, 2017 1 commit
    • Julien Muchembled's avatar
      Review Connection/DB closure · 1b9475d4
      Julien Muchembled authored
      In addition to some micro-optimisation, this fixes the following minor issues:
      - Closing a DB left Connections registered to the global transaction manager.
        Which broke at least multi-db because only such primary connections were
        ignored with monkey-patches.
      - Stop calling the real newTransaction when aborting an existing transaction.
        This was not the case if explicit_transactions=0.
      1b9475d4
  10. 07 Nov, 2017 1 commit
  11. 30 Oct, 2017 1 commit
  12. 27 Oct, 2017 2 commits
  13. 05 Oct, 2017 1 commit
  14. 30 Aug, 2017 2 commits
  15. 29 Aug, 2017 2 commits
    • Jason Madden's avatar
      Use a higher pickle protocol for serializing objects on Python 2 (#179) · be5a9d54
      Jason Madden authored
      * Use a higher pickle protocol (2) for serializing objects on Python 2
      
      Previously protocol 1 was used. This is more efficient for new-style
      classes (all persistent objects are new-style), according to the docs,
      at the cost of being very slightly less space efficient for old-style
      classes.
      
      In tests of a persistent object with two trivial numeric attributes,
      the higher protocol was 12 bytes smaller, and serialized and
      deserialized 1us faster. Introducing a reference to another new-style
      class for a more realistic test made the higher protocol twice as fast
      to serialize (20.5 vs 10.3us), almost half the size (215 vs 142
      bytes), and it deserialized 30% faster (6.5 vs 4.6us).
      
      On Python 2, this will now allow open ``file`` objects to be
      pickled (loading the object will result in a closed file); previously
      this would result in a ``TypeError`` (as does under Python 3). We had
      tests that you couldn't do that with a BlobFile so I had to update it
      to still make that true.
      
      I wouldn't recommend serializing arbitrary open files under Python
      2 (for one thing, they can't trivially be deserialized in Python 3),
      but I didn't take any steps to prevent it either. Since this hasn't
      been possible, there shouldn't be code in the wild that is trying to
      do it---and it wouldn't be forward compatible with Python 3 either.
      be5a9d54
    • XeL64's avatar
      fix typo (#180) · b340b651
      XeL64 authored
      b340b651
  16. 25 Jul, 2017 4 commits
  17. 24 Jul, 2017 11 commits
  18. 17 May, 2017 5 commits
  19. 17 Apr, 2017 1 commit
    • Jason Madden's avatar
      Make all classes new-style. (#160) · e714f515
      Jason Madden authored
      * Make all classes new-style.
      
      On PyPy, it's documented (http://pypy.org/performance.html#id3) that
      old-style classes are slow, and classes that derive from both old and
      new are especially slow. I checked with the PyPy devs on IRC today and
      they confirmed that's still true with the latest PyPy2 releases.
      
      Unfortunately, FileStorage was one such mixed class, as was Connection.
      
      Moving to new-style classes seems to have a positive impact in the
      benchmarks. Here's zodbshootout on PyPy2 5.7.1 against a
      FileStorage (running in --threads and with 5 reps to be sure we get
      warmed up). First, current ZODB:
      
      "Transaction",                fs
      "Add 1000 Objects",            31,738
      "Update 1000 Objects",         42,444
      "Read 1000 Cold Objects",      51,894
      "Read 1000 Hot Objects",       53,187
      "Read 1000 Steamin' Objects", 835,877
      
      And with this PR:
      
      "Transaction",                fs
      "Add 1000 Objects",             35,651
      "Update 1000 Objects",          54,906
      "Read 1000 Cold Objects",      103,484
      "Read 1000 Hot Objects",        84,721
      "Read 1000 Steamin' Objects", 2,112,095
      
      The tests that hit the storage extensively are notably faster, as are
      steamin and hot, Connection having been a mixed class.
      
      I ran all tests multiple times. The data files were removed between
      runs. There's some variation, but the new-style classes always seem
      better.
      
      For comparison, here's CPython 2.7.13:
      
      "Transaction",                fs
      "Add 1000 Objects",            19,531
      "Update 1000 Objects",         16,201
      "Read 1000 Cold Objects",      22,111
      "Read 1000 Hot Objects",       21,851
      "Read 1000 Steamin' Objects", 880,582
      
      Locally I've run the tests under 2.7 and they all passed.
      e714f515