1. 18 Mar, 2013 3 commits
  2. 13 Mar, 2013 1 commit
  3. 08 Mar, 2013 2 commits
  4. 05 Mar, 2013 1 commit
    • Marius Gedminas's avatar
      Python 3: pickle bytestrings using SHORT_BINSTRING · a00d35fe
      Marius Gedminas authored
      This uses bytes_as_strings=True option introduced in zodbpickle 0.2 for
      this purpose.
      
      This way pickles produced on Python 3 are nearly the same as on Python 2.
      There are some slight differences (Python 3 seems to perform more
      memoizations which grows the size of some pickles by a couple of bytes),
      but they're immaterial.
      
      Now we can use zodbpickle's noload() on Python 3 to scan pickles for
      persistent references.  We couldn't do that before, because Python 3
      normally pickles byte strings as calls to codecs.encode(u'latin1-data',
      'latin-1'), and noload() doesn't interpret the REDUCE opcode involved in
      that representation.
      
      Note that when you're pickling byte strings using bytes_as_strings=True,
      you have to load them using encoding='bytes' (which breaks instances, so
      cannot be used) or using errors='bytes' (which mean some bytestrings may
      get unpickled as unicode instead).  I've tried hard to discover every
      place that unpickles OIDs and added conversion to bytes in those places.
      
      Applications dealing with binary data be prepared to handle bytestrings
      that unexpectedly become unicode on unpickling.  That's the price of
      Python 2 compatibility.
      a00d35fe
  5. 03 Mar, 2013 3 commits
  6. 02 Mar, 2013 2 commits
  7. 01 Mar, 2013 5 commits
  8. 28 Feb, 2013 16 commits
  9. 27 Feb, 2013 5 commits
  10. 26 Feb, 2013 1 commit
  11. 20 Feb, 2013 1 commit
    • Marius Gedminas's avatar
      Normalize fstail output on Python 3 · 2f429e61
      Marius Gedminas authored
      The only remaining test failures are due to changed pickle sizes.  I've
      discovered two reasons for that so far:
      
        1) Python 3 pickles 'native strings' as BINUNICODE, Python 2 pickles
           'native strings' as SHORT_BINSTRING, which is 3 bytes shorter.
      
        2) Python 3 and also the 'pickle' module Python 2 pickles tuples like
           (UserDefinedClass, ) with an extra BINPUT opcode at the end,
           compared to 'cPickle' in Python 2 when given the same tuple:
      
              Python 2.7.3 (default, Sep 26 2012, 21:51:14)
              [GCC 4.7.2] on linux2
              Type "help", "copyright", "credits" or "license" for more information.
              >>> import pickle, cPickle
              >>> class MyClass(object): pass
              ...
              >>> len(pickle.dumps((MyClass, None), 1))
              26
              >>> len(cPickle.dumps((MyClass, None), 1))
              24
      
      The plan is to switch to zodbpickle instead of the builtin Python 3
      pickle, then use encoding=bytes from http://bugs.python.org/issue6784
      and fix the class-in-tuple encoding to match Python 2's cPickle.
      2f429e61