• Kirill Smelkov's avatar
    wcfs: tests: Add test do demonstrate "at out of bounds" crash on readPinWatchers -> ΔFtail.BlkRevAt · 97ce5105
    Kirill Smelkov authored
    The codepath that sends pin messages to watchers on FUSE READ, similarly
    to what was showed in 339f1884 is also vulnerable to "at out of bounds"
    panic if at=ΔFtail.tail:
    
        wcfs_test.py::test_wcfs_crash_old_data
        ---------------- live log call -----------------
        WARNING  ZODB.FileStorage:FileStorage.py:413 Ignoring index for /tmp/testdb_fs.nbSKXu/1.fs
    
        M: commit -> @at0 (03e5a31e5e5ef6bb)
    
        M: commit -> @at1 (03e5a31e5e63fa77)
        M:      f<0000000000000002>     [0]
        INFO     wcfs:__init__.py:293 starting for file:///tmp/testdb_fs.nbSKXu/1.fs ...
        I0120 16:50:22.136098  697106 wcfs.go:2393] start "/dev/shm/wcfs/93026d44ef96f87df2cc0e2e451c5aabee91b652" "file:///tmp/testdb_fs.nbSKXu/1.fs"
        I0120 16:50:22.136127  697106 wcfs.go:2399] (built with go1.17.6)
        W0120 16:50:22.136233  697106 storage.go:152] zodb: FIXME: open file:///tmp/testdb_fs.nbSKXu/1.fs: raw cache is not ready for invalidations -> NoCache forced
        INFO     wcfs:__init__.py:334 started pid697106 @ /dev/shm/wcfs/93026d44ef96f87df2cc0e2e451c5aabee91b652
    
        C: setup watch f<0000000000000002> @at1 (03e5a31e5e63fa77)
        #  pinok: {}
        panic: at out of bounds: at: @03e5a31e5e63fa77,  (tail, head] = (@03e5a31e5e63fa77, @03e5a31e5e63fa77]
    
        goroutine 7 [running]:
        lab.nexedi.com/nexedi/wendelin.core/wcfs/internal/zdata.panicf(...)
                /home/kirr/src/neo/src/lab.nexedi.com/nexedi/wendelin.core/wcfs/internal/zdata/misc.go:47
        lab.nexedi.com/nexedi/wendelin.core/wcfs/internal/zdata.(*ΔFtail).BlkRevAt(0xc0000a5d40, {0x969718, 0xc000076140}, 0xc0001a22a0, 0xc0001c0200, 0x3e5a31e5e63fa77)
                /home/kirr/src/neo/src/lab.nexedi.com/nexedi/wendelin.core/wcfs/internal/zdata/δftail.go:1077 +0xa45
        main.(*BigFile).readPinWatchers(0xc0001d0200, {0x969718, 0xc000076140}, 0x0, 0xffffffffffffffff)
                /home/kirr/src/neo/src/lab.nexedi.com/nexedi/wendelin.core/wcfs/wcfs.go:1559 +0x2a5
        main.(*BigFile).readBlk(0xc0001d0200, {0x969718, 0xc000076140}, 0x0, {0xc000320000, 0x200000, 0x0})
                /home/kirr/src/neo/src/lab.nexedi.com/nexedi/wendelin.core/wcfs/wcfs.go:1281 +0x4d2
        main.(*BigFile).Read.func1({0x969718, 0xc000076140})
                /home/kirr/src/neo/src/lab.nexedi.com/nexedi/wendelin.core/wcfs/wcfs.go:1223 +0x71
        lab.nexedi.com/kirr/go123/xsync.(*WorkGroup).Go.func1()
                /home/kirr/src/neo/src/lab.nexedi.com/kirr/go123/xsync/xsync.go:86 +0x68
        created by lab.nexedi.com/kirr/go123/xsync.(*WorkGroup).Go
                /home/kirr/src/neo/src/lab.nexedi.com/kirr/go123/xsync/xsync.go:83 +0x92
        >>> Change history by file:
    
        f<0000000000000002>:
                                        0 1 2 3 4 5 6 7
                                        a b c d e f g h
                @at0 (03e5a31e5e5ef6bb)
                @at1 (03e5a31e5e63fa77) 0
    
        ...
    
            @func
            def test_wcfs_crash_old_data():
                # start wcfs with ΔFtail/ΔBtail not covering that initial data.
                t = tDB(old_data=[{0:'a'}]); zf = t.zfile; at1 = t.head
                defer(t.close)
    
                f = t.open(zf)
    
                # ΔFtail coverage is currently (at1,at1]
                wl = t.openwatch()
                wl.watch(zf, at1, {})
    
                # wcfs is crashing on readPinWatcher -> ΔFtail.BlkRevAt with
                #   "at out of bounds: at: @at1,  (tail,head] = (@at1,@at1]
                # because BlkRevAt(at=tail) query was disallowed.
        >       f.assertBlk(0, 'a')          # [0] becomes tracked
    
    Still also crashing in test_wcfs_watch_setup.
    97ce5105
wcfs_test.py 68.2 KB