1. 14 Jul, 2020 18 commits
    • Kirill Smelkov's avatar
      go/zodb/zeo: Add support for invalidations · d3a0bccb
      Kirill Smelkov authored
      Receive invalidation from server and send corresponding events to watchq.
      Care to send only events with tid > at0 that we initially returned when opening.
      
      Tests pass, but they need https://github.com/zopefoundation/ZEO/pull/160
      d3a0bccb
    • Kirill Smelkov's avatar
      go/zodb/zeo: Teach zlink to serve incoming notification and requests · 2aa27ef9
      Kirill Smelkov authored
      This will be used in the next patch to handle invalidateTransaction
      messages.
      2aa27ef9
    • Kirill Smelkov's avatar
      go/zodb/zeo: MsgPack support · 63d322b9
      Kirill Smelkov authored
      ZEO5 adds way for messages to be encoded via either pickles or MessagePack.
      However until now we were always using pickles.
      Let's add msgpack support to be able to e.g. use wire encoding that
      server prefers.
      
      MsgPack support is almost fully localized in encoding.
      We use tinylib/msgp runtime routines to decode/encode msg fields with known types,
      and shamaton/msgpack to decode/encode msg.arg, which is interface{},
      because msgp does not generally work for arbitrary reflections.
      
      For msgpack=true, tests state is the same as with pickles: handshake
      works, but load fails when verifying that Load returns correct error for
      deleted object:
      
          TestLoad/py/msgpack=false: xtesting.go:272: load 0285cbacc06d3a4c:0000000000000007: returned err unexpected:
              have: /tmp/zeo170183943/1.fs.zeosock: load 0285cbacc06d3a4c:0000000000000007: 0000000000000007: no such object
              want: /tmp/zeo170183943/1.fs.zeosock: load 0285cbacc06d3a4c:0000000000000007: 0000000000000007: object was deleted @0285cbacc06d3a4c
          TestLoad/py/msgpack=false: xtesting.go:272: load 0285cbad858bf2e6:0000000000000006: returned err unexpected:
              have: /tmp/zeo170183943/1.fs.zeosock: load 0285cbad858bf2e6:0000000000000006: 0000000000000006: no such object
              want: /tmp/zeo170183943/1.fs.zeosock: load 0285cbad858bf2e6:0000000000000006: 0000000000000006: object was deleted @0285cbad858bf2e6
          TestLoad/py/msgpack=false: xtesting.go:290: load 7fffffffffffffff:0000000000000007: returned err unexpected:
              have: /tmp/zeo170183943/1.fs.zeosock: load 7fffffffffffffff:0000000000000007: 0000000000000007: no such object
              want: /tmp/zeo170183943/1.fs.zeosock: load 7fffffffffffffff:0000000000000007: 0000000000000007: object was deleted @0285cbacc06d3a4c
          TestLoad/py/msgpack=false: xtesting.go:290: load 7fffffffffffffff:0000000000000006: returned err unexpected:
              have: /tmp/zeo170183943/1.fs.zeosock: load 7fffffffffffffff:0000000000000006: 0000000000000006: no such object
              want: /tmp/zeo170183943/1.fs.zeosock: load 7fffffffffffffff:0000000000000006: 0000000000000006: object was deleted @0285cbad858bf2e6
      
          TestLoad/py/msgpack=true: xtesting.go:272: load 0285cbacc06d3a4c:0000000000000007: returned err unexpected:
              have: /tmp/zeo247652538/1.fs.zeosock: load 0285cbacc06d3a4c:0000000000000007: 0000000000000007: no such object
              want: /tmp/zeo247652538/1.fs.zeosock: load 0285cbacc06d3a4c:0000000000000007: 0000000000000007: object was deleted @0285cbacc06d3a4c
          TestLoad/py/msgpack=true: xtesting.go:272: load 0285cbad858bf2e6:0000000000000006: returned err unexpected:
              have: /tmp/zeo247652538/1.fs.zeosock: load 0285cbad858bf2e6:0000000000000006: 0000000000000006: no such object
              want: /tmp/zeo247652538/1.fs.zeosock: load 0285cbad858bf2e6:0000000000000006: 0000000000000006: object was deleted @0285cbad858bf2e6
          TestLoad/py/msgpack=true: xtesting.go:290: load 7fffffffffffffff:0000000000000007: returned err unexpected:
              have: /tmp/zeo247652538/1.fs.zeosock: load 7fffffffffffffff:0000000000000007: 0000000000000007: no such object
              want: /tmp/zeo247652538/1.fs.zeosock: load 7fffffffffffffff:0000000000000007: 0000000000000007: object was deleted @0285cbacc06d3a4c
          TestLoad/py/msgpack=true: xtesting.go:290: load 7fffffffffffffff:0000000000000006: returned err unexpected:
              have: /tmp/zeo247652538/1.fs.zeosock: load 7fffffffffffffff:0000000000000006: 0000000000000006: no such object
              want: /tmp/zeo247652538/1.fs.zeosock: load 7fffffffffffffff:0000000000000006: 0000000000000006: object was deleted @0285cbad858bf2e6
      
      This is due to https://github.com/zopefoundation/ZODB/issues/318
      63d322b9
    • Kirill Smelkov's avatar
      go/zodb/zeo: Accept both int|bool for pickled message flags · f7543195
      Kirill Smelkov authored
      It is already documented in pktDecodeZ comment that flags is int|bool.
      However until now we were decoding it only as int.
      
      As is it was working, but it will fail when receiving e.g.
      invalidateTransaction message, because ZEO/py server actually uses bool
      when sending it:
      
          https://github.com/zopefoundation/ZEO/blob/5.2.1-20-gcb26281d/src/ZEO/asyncio/base.py#L139
      
      Fix it.
      This will be covered by watch tests, when watch support will be added in a followup patch.
      f7543195
    • Kirill Smelkov's avatar
      go/zodb/zeo: Factor-out tuple handling into encoding · fe24c62f
      Kirill Smelkov authored
      Handle pickled lists as valid input when decoding tuples.
      fe24c62f
    • Kirill Smelkov's avatar
    • Kirill Smelkov's avatar
      go/zodb/zeo: proto: Use enc.<X> and enc.as<X> to encode/decode type X · ffe0d6e9
      Kirill Smelkov authored
      This is more uniform and will be followed by all data types in the next patch.
      Here rename tid/oid pack/unpack routines correspondingly.
      Add docstrings for X=tid|oid.
      ffe0d6e9
    • Kirill Smelkov's avatar
      go/zodb/zeo: Introduce notion of encoding · 0696cd64
      Kirill Smelkov authored
      Keep information about which message encoding is used on the wire in
      encoding type. Make pktDecode/pktEncode and data type conversion
      utilities be methods of this type. For now there is only 'pickles'
      encoding, but we'll soon introduce 'msgpack'.
      
      Currently not everything related to pickles is localized in encoding -
      we'll be moving more bits to encoding in the followup patches.
      0696cd64
    • Kirill Smelkov's avatar
    • Kirill Smelkov's avatar
      go/zodb/zeo: Start moving things related to protocol and messages encoding into proto.go · 362e555a
      Kirill Smelkov authored
      First step:
      
      - move msg and msgFlags
      - move pktDecode
      - move functions to pack/unpack tid and oid
      
      more to come.
      362e555a
    • Kirill Smelkov's avatar
      go/zodb/zeo: Shutdown zlink on first error · e7ae218b
      Kirill Smelkov authored
      The same behaviour as ZEO/py does.
      e7ae218b
    • Kirill Smelkov's avatar
      go/zodb/zeo: Use zlink as term for link in between peers · b82154d5
      Kirill Smelkov authored
      Just renaming.
      b82154d5
    • Kirill Smelkov's avatar
      go/zodb/zeo: Don't avoid defer · 5a48e702
      Kirill Smelkov authored
      Starting from Go1.14 defer is no longer slow:
      https://golang.org/doc/go1.14#runtime
      5a48e702
    • Kirill Smelkov's avatar
      8889743e
    • Kirill Smelkov's avatar
      go/zodb/zeo: Initial tests · 589faf07
      Kirill Smelkov authored
      As promised in b65f6d0f (go/zodb: Teach ZODB/go to access ZEO (draft))
      let's start to add tests for ZEO/go client.
      
      The client is tested against ZEO/py server.
      Cover handshake and Load.
      
      Load currently fails beause in ZODB/py POSKeyError does not allow to
      distinguish whether it is no object at all or object exists and its data
      was not found for tid_before.
      
          --- FAIL: TestLoad/py (0.22s)
              zeo_test.go:217: load 0285cbacc06d3a4c:0000000000000007: returned err unexpected:
                  have: /tmp/zeo273044293/1.fs.zeosock: load 0285cbacc06d3a4c:0000000000000007: 0000000000000007: no such object
                  want: /tmp/zeo273044293/1.fs.zeosock: load 0285cbacc06d3a4c:0000000000000007: 0000000000000007: object was deleted @0285cbacc06d3a4c
              zeo_test.go:217: load 0285cbad858bf2e6:0000000000000006: returned err unexpected:
                  have: /tmp/zeo273044293/1.fs.zeosock: load 0285cbad858bf2e6:0000000000000006: 0000000000000006: no such object
                  want: /tmp/zeo273044293/1.fs.zeosock: load 0285cbad858bf2e6:0000000000000006: 0000000000000006: object was deleted @0285cbad858bf2e6
              zeo_test.go:217: load 7fffffffffffffff:0000000000000006: returned err unexpected:
                  have: /tmp/zeo273044293/1.fs.zeosock: load 7fffffffffffffff:0000000000000006: 0000000000000006: no such object
                  want: /tmp/zeo273044293/1.fs.zeosock: load 7fffffffffffffff:0000000000000006: 0000000000000006: object was deleted @0285cbad858bf2e6
              zeo_test.go:217: load 7fffffffffffffff:0000000000000007: returned err unexpected:
                  have: /tmp/zeo273044293/1.fs.zeosock: load 7fffffffffffffff:0000000000000007: 0000000000000007: no such object
                  want: /tmp/zeo273044293/1.fs.zeosock: load 7fffffffffffffff:0000000000000007: 0000000000000007: object was deleted @0285cbacc06d3a4c
      
      Problem reported upstream:
      
      -> https://github.com/zopefoundation/ZODB/issues/318
      589faf07
    • Kirill Smelkov's avatar
      go/zodb/zeo: Fix thinko in openByURL error context · 24c9f49f
      Kirill Smelkov authored
      No need to use trailing : suffix as xerr.Contextf automatically adds one.
      24c9f49f
    • Kirill Smelkov's avatar
      go/internal/xtesting: LoadDBHistory · e7a65f87
      Kirill Smelkov authored
      This is utility to load whole content of a ZODB database.
      
      It will be useful to load data to expect from e.g. dataok.fs and feed it
      as txnvOk to DrvTestLoad when verifying another storage driver.
      e7a65f87
    • Kirill Smelkov's avatar
      go/internal/xtesting: FatalIf · 41240d2e
      Kirill Smelkov authored
      This is utility to create X function, similar to exc.Raisif, but that
      would call t.Fatal instead.
      41240d2e
  2. 10 Jul, 2020 3 commits
    • Kirill Smelkov's avatar
      go/zodb/fs1: tests: Factor-out infrastructure to verify Watch into common place · 13c77191
      Kirill Smelkov authored
      We will soon need this infrastructure to test other storage drivers.
      13c77191
    • Kirill Smelkov's avatar
      go/internal/xtesting: DrvTestLoad: More easier to read error messages · 88d0165f
      Kirill Smelkov authored
      E.g. before:
      
          --- FAIL: TestLoad/py (0.22s)
              zeo_test.go:217: load 0285cbacc06d3a4c:0000000000000007: returned err unexpected: /tmp/zeo484021674/1.fs.zeosock: load 0285cbacc06d3a4c:0000000000000007: 0000000000000007: no such object  ; want: /tmp/zeo484021674/1.fs.zeosock: load 0285cbacc06d3a4c:0000000000000007: 0000000000000007: object was deleted @0285cbacc06d3a4c
              zeo_test.go:217: load 0285cbad858bf2e6:0000000000000006: returned err unexpected: /tmp/zeo484021674/1.fs.zeosock: load 0285cbad858bf2e6:0000000000000006: 0000000000000006: no such object  ; want: /tmp/zeo484021674/1.fs.zeosock: load 0285cbad858bf2e6:0000000000000006: 0000000000000006: object was deleted @0285cbad858bf2e6
              zeo_test.go:217: load 7fffffffffffffff:0000000000000006: returned err unexpected: /tmp/zeo484021674/1.fs.zeosock: load 7fffffffffffffff:0000000000000006: 0000000000000006: no such object  ; want: /tmp/zeo484021674/1.fs.zeosock: load 7fffffffffffffff:0000000000000006: 0000000000000006: object was deleted @0285cbad858bf2e6
              zeo_test.go:217: load 7fffffffffffffff:0000000000000007: returned err unexpected: /tmp/zeo484021674/1.fs.zeosock: load 7fffffffffffffff:0000000000000007: 0000000000000007: no such object  ; want: /tmp/zeo484021674/1.fs.zeosock: load 7fffffffffffffff:0000000000000007: 0000000000000007: object was deleted @0285cbacc06d3a4c
      
      After:
      
          --- FAIL: TestLoad/py (0.22s)
              zeo_test.go:217: load 0285cbacc06d3a4c:0000000000000007: returned err unexpected:
                  have: /tmp/zeo087978981/1.fs.zeosock: load 0285cbacc06d3a4c:0000000000000007: 0000000000000007: no such object
                  want: /tmp/zeo087978981/1.fs.zeosock: load 0285cbacc06d3a4c:0000000000000007: 0000000000000007: object was deleted @0285cbacc06d3a4c
              zeo_test.go:217: load 0285cbad858bf2e6:0000000000000006: returned err unexpected:
                  have: /tmp/zeo087978981/1.fs.zeosock: load 0285cbad858bf2e6:0000000000000006: 0000000000000006: no such object
                  want: /tmp/zeo087978981/1.fs.zeosock: load 0285cbad858bf2e6:0000000000000006: 0000000000000006: object was deleted @0285cbad858bf2e6
              zeo_test.go:217: load 7fffffffffffffff:0000000000000007: returned err unexpected:
                  have: /tmp/zeo087978981/1.fs.zeosock: load 7fffffffffffffff:0000000000000007: 0000000000000007: no such object
                  want: /tmp/zeo087978981/1.fs.zeosock: load 7fffffffffffffff:0000000000000007: 0000000000000007: object was deleted @0285cbacc06d3a4c
              zeo_test.go:217: load 7fffffffffffffff:0000000000000006: returned err unexpected:
                  have: /tmp/zeo087978981/1.fs.zeosock: load 7fffffffffffffff:0000000000000006: 0000000000000006: no such object
                  want: /tmp/zeo087978981/1.fs.zeosock: load 7fffffffffffffff:0000000000000006: 0000000000000006: object was deleted @0285cbad858bf2e6
      88d0165f
    • Kirill Smelkov's avatar
      go/zodb/fs1: tests: Factor-out infrastructure to verify Load into common place · 9b584e08
      Kirill Smelkov authored
      We will soon need this infrastructure to test other storage drivers.
      9b584e08
  3. 07 Jul, 2020 7 commits
  4. 09 Jun, 2020 1 commit
  5. 05 Apr, 2020 2 commits
  6. 14 Feb, 2020 1 commit
  7. 10 Feb, 2020 4 commits
  8. 16 Jan, 2020 1 commit
  9. 12 Apr, 2019 1 commit
  10. 12 Mar, 2019 2 commits