• Kirill Smelkov's avatar
    wcfs: tests: Use bytestrings uniformly · e1c96b1d
    Kirill Smelkov authored
    WCFS tests follow bytestring model from the beginning because it
    operates with binary data and binary messages to/from WCFS server. It
    all works ok on py2, but running tests on py3 yielded several problems.
    For example str and bytes were mixed in the innermost part of _assertBlk
    
            def _(ctx, ev):
                assert t.cached()[blk] == cached
                ev.append('read pre')
    
                # access data with released GIL so that the thread that reads data from
                # head/watch can receive pin message. Be careful to handle cancellation,
                # so that on error in another worker we don't get stuck and the
                # error can be propagated to wait and reported.
                #
                # we handle cancellation by spawning read in another thread and
                # waiting for either ctx cancel, or read thread to complete. This
                # way on ctx cancel (e.g. assertion failure in another worker), the
                # read thread can remain running even after _assertBlk returns, and
                # in particular till the point where the whole test is marked as
                # failed and shut down. But on test shutdown .fmmap is unmapped for
                # all opened tFiles, and so read will hit SIGSEGV. Prepare to catch
                # that SIGSEGV here.
                have_read = chan(1)
                def _():
                    try:
                        b = read_exfault_nogil(blkview[0:1])
                    except SegmentationFault:
                        b = 'FAULT'
                    t._blkaccess(blk)
                    have_read.send(b)
                go(_)
                _, _rx = select(
                    ctx.done().recv,    # 0
                    have_read.recv,     # 1
                )
                if _ == 0:
                    raise ctx.err()
                b = _rx
    
        >       ev.append('read ' + b)
        E       TypeError: can only concatenate str (not "bytes") to str
    
    bytes input was split with str delimiter
    
        def _loadStats(t): # -> {}
            stats = {}
            for l in t.wc._read(".wcfs/stats").splitlines():
                # key : value
    >           k, v = l.split(':')
    E           TypeError: a bytes-like object is required, not 'str'
    
    and str object rejected when assigning to C `char*`:
    
        Traceback (most recent call last):
          File "golang/_golang.pyx", line 156, in golang._golang.__goviac
          File "wcfs/internal/wcfs_test.pyx", line 58, in wendelin.wcfs.internal.wcfs_test._tWCFS._abort_ontimeout
        TypeError: expected bytes, str found
    
    -> Fix all those overlooks by consistently using bytestrings everywhere.
    
    On py3 the implementation depends on nexedi/pygolang!21,
    but on py2 it works both with and without pygolang bstr patches.
    
    Preliminary history:
    
        vnmabus/wendelin.core@af56bd31
    
    but it takes the reverse approach and mixes in pytest.approx for
    timestamp assert which is unrelated to the topic and is not correct as
    default pytest.approx behaviour is to use 1e-6 relative precision which
    results in ~1700s seconds tolerance instead of intended 1µs:
    
        In [1]: import pytest
        In [2]: import time
    
        In [3]: t = time.time()
    
        In [4]: t
        Out[4]: 1727271692.0636573
    
        In [5]: t == t
        Out[5]: True
    
        In [6]: t == pytest.approx(t+1000)
        Out[6]: True
    
        In [7]: t == pytest.approx(t+2000)
        Out[7]: False
    
    So using pytest.approx resulted in tests to become accepting faulty wcfs
    behaviour.
    
    Timestamp assert will be handled properly in the next patch.
    Co-authored-by: Carlos Ramos Carreño's avatarCarlos Ramos Carreño <carlos.ramos@nexedi.com>
    e1c96b1d
wcfs_test.py 74 KB