• Kirill Smelkov's avatar
    commit: Fix it wrt ZEO · 858c134c
    Kirill Smelkov authored
    Since 7ae5ff82 (Port zodbtools to py3) zodbtools/py3 requires
    pygolang!21 with which
    running `zodb commit` wrt ZEO, as demonstrates by added
    test_zodbcommit_cmd, yields:
    
        test/test_commit.py::test_zodbcommit_cmd[ZEO-!zext] Traceback (most recent call last):
          File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
            "__main__", fname, loader, pkg_name)
          File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
            exec code in run_globals
          File "/home/kirr/src/wendelin/z/zodbtools/zodbtools/zodb.py", line 133, in <module>
            main()
          File "/home/kirr/src/wendelin/z/zodbtools/zodbtools/zodb.py", line 129, in main
            return command_module.main(argv)
          File "<decorator-gen-4>", line 2, in main
          File "/home/kirr/src/tools/go/pygolang/golang/__init__.py", line 125, in _
            return f(*argv, **kw)
          File "/home/kirr/src/wendelin/z/zodbtools/zodbtools/zodbcommit.py", line 232, in main
            tid = zodbcommit(stor, at, txn)
          File "/home/kirr/src/wendelin/z/zodbtools/zodbtools/zodbcommit.py", line 131, in zodbcommit
            _()
          File "/home/kirr/src/wendelin/z/zodbtools/zodbtools/zodbcommit.py", line 128, in _
            stor.store(obj.oid, current_serial(obj.oid), data, '', txn)
          File "/home/kirr/src/wendelin/z/zodbtools/zodbtools/zodbcommit.py", line 87, in current_serial
            return _serial_at(stor, oid, at)
          File "/home/kirr/src/wendelin/z/zodbtools/zodbtools/zodbcommit.py", line 152, in _serial_at
            xdata = stor.loadBefore(oid, before)
          File "/home/kirr/src/wendelin/z/ZEO5/src/ZEO/ClientStorage.py", line 616, in loadBefore
            return self._server.load_before(oid, tid)
          File "/home/kirr/src/wendelin/z/ZEO5/src/ZEO/asyncio/client.py", line 935, in load_before
            return self.io_call(self.load_before_co(oid, tid, self.timeout))
          File "/home/kirr/src/wendelin/z/ZEO5/src/ZEO/asyncio/client.py", line 862, in io_call
            return future.result() if wait else future
          File "/home/kirr/src/wendelin/z/ZEO5/src/ZEO/asyncio/futures.py", line 230, in result
            return Future.result(self)
          File "/home/kirr/src/wendelin/z/ZEO5/src/ZEO/asyncio/futures.py", line 81, in result
            raise self._result
        ClientDisconnected: [Errno 104] Connection reset by peer
        FAILED()
        zeo.log:
    
        2024-07-16T21:13:45 ERROR ZEO.asyncio.server Can't deserialize message
        Traceback (most recent call last):
          File "/home/kirr/src/wendelin/z/ZEO5/src/ZEO/asyncio/server.py", line 100, in message_received
            message_id, async_, name, args = self.decode(message)
          File "/home/kirr/src/wendelin/z/ZEO5/src/ZEO/asyncio/marshal.py", line 119, in pickle_server_decode
            return unpickler.load()  # msgid, flags, name, args
          File "/home/kirr/src/wendelin/z/ZEO5/src/ZEO/asyncio/marshal.py", line 176, in server_find_global
            raise ImportError("Module not allowed: %s" % (module,))
        ImportError: Module not allowed: golang
    
    This happens because zodbcommit.main and zodbdump.DumpReader use bstr to
    handle binary data and further pass as IStorageTransactionInformation
    instance to ZEO via its RPC, but ZEO server explicitly checks safe to
    import modules and rejects golang.
    
    One way to fix would be to patch ZEO to allow golang imports, but in
    general it is better to follow the interface strictly and stay 100%
    compatible with its users. That's why the fix converts fields covered by
    IStorageTransactionInformation to exact types specified by that
    interface to stay 100% compatible with users of the interface because on
    py3 e.g. (b' ' == ' ') returns False and so we need to be careful to
    provide .status as exactly str instead bytes, and do the similar for
    other fields.
    858c134c
test_commit.py 4.68 KB