• Kirill Smelkov's avatar
    encoder: Allow to specify pickle protocol version · 06e06939
    Kirill Smelkov authored
    There are many pickle protocol versions - 0 to 4. Python2 for example
    understands only versions 0 - 2. However we currently unconditionally
    emit opcodes from higher versions, for example STACK_GLOBAL - from
    version 4 - when encoding a Class, which leads to inability to decode
    pickles generated by ogórek on Python2.
    
    Similarly protocol 0 states that only text opcodes should be used,
    however we currently unconditionally emit e.g. BININT (from protocol 1)
    when encoding integers.
    
    Changing to always using protocol 0 opcodes would be not good, since many
    opcodes for efficiently encoding either integers, booleans, unicode etc
    are available only in protocol versions 2 and 4.
    
    For this reason, similarly to Python[1], let's allow users to specify
    desired pickle protocol when creating Encoder with config. For backward
    compatibility and common sense the protocol version that plain
    NewEncoder selects is 2.
    
    This commit adds only above-described user interface and testing
    infrastructure for verifying what was the result of encoding an object
    at particular protocol version.
    
    For now only a few of pickle test vectors are right wrt what the encoder
    should be or currently generates. Thus in the next patches we'll be
    step-by-step fixing encoder on this topic.
    
    [1] https://docs.python.org/3/library/pickle.html#pickle.dump
    06e06939
encode.go 8.75 KB