wcfs: tests: Add test do demonstrate "at out of bounds" crash on readPinWatchers -> ΔFtail.BlkRevAt

The codepath that sends pin messages to watchers on FUSE READ, similarly
to what was showed in 339f1884 is also vulnerable to "at out of bounds"
panic if at=ΔFtail.tail:

    wcfs_test.py::test_wcfs_crash_old_data
    ---------------- live log call -----------------
    WARNING  ZODB.FileStorage:FileStorage.py:413 Ignoring index for /tmp/testdb_fs.nbSKXu/1.fs

    M: commit -> @at0 (03e5a31e5e5ef6bb)

    M: commit -> @at1 (03e5a31e5e63fa77)
    M:      f<0000000000000002>     [0]
    INFO     wcfs:__init__.py:293 starting for file:///tmp/testdb_fs.nbSKXu/1.fs ...
    I0120 16:50:22.136098  697106 wcfs.go:2393] start "/dev/shm/wcfs/93026d44ef96f87df2cc0e2e451c5aabee91b652" "file:///tmp/testdb_fs.nbSKXu/1.fs"
    I0120 16:50:22.136127  697106 wcfs.go:2399] (built with go1.17.6)
    W0120 16:50:22.136233  697106 storage.go:152] zodb: FIXME: open file:///tmp/testdb_fs.nbSKXu/1.fs: raw cache is not ready for invalidations -> NoCache forced
    INFO     wcfs:__init__.py:334 started pid697106 @ /dev/shm/wcfs/93026d44ef96f87df2cc0e2e451c5aabee91b652

    C: setup watch f<0000000000000002> @at1 (03e5a31e5e63fa77)
    #  pinok: {}
    panic: at out of bounds: at: @03e5a31e5e63fa77,  (tail, head] = (@03e5a31e5e63fa77, @03e5a31e5e63fa77]

    goroutine 7 [running]:
    lab.nexedi.com/nexedi/wendelin.core/wcfs/internal/zdata.panicf(...)
            /home/kirr/src/neo/src/lab.nexedi.com/nexedi/wendelin.core/wcfs/internal/zdata/misc.go:47
    lab.nexedi.com/nexedi/wendelin.core/wcfs/internal/zdata.(*ΔFtail).BlkRevAt(0xc0000a5d40, {0x969718, 0xc000076140}, 0xc0001a22a0, 0xc0001c0200, 0x3e5a31e5e63fa77)
            /home/kirr/src/neo/src/lab.nexedi.com/nexedi/wendelin.core/wcfs/internal/zdata/δftail.go:1077 +0xa45
    main.(*BigFile).readPinWatchers(0xc0001d0200, {0x969718, 0xc000076140}, 0x0, 0xffffffffffffffff)
            /home/kirr/src/neo/src/lab.nexedi.com/nexedi/wendelin.core/wcfs/wcfs.go:1559 +0x2a5
    main.(*BigFile).readBlk(0xc0001d0200, {0x969718, 0xc000076140}, 0x0, {0xc000320000, 0x200000, 0x0})
            /home/kirr/src/neo/src/lab.nexedi.com/nexedi/wendelin.core/wcfs/wcfs.go:1281 +0x4d2
    main.(*BigFile).Read.func1({0x969718, 0xc000076140})
            /home/kirr/src/neo/src/lab.nexedi.com/nexedi/wendelin.core/wcfs/wcfs.go:1223 +0x71
    lab.nexedi.com/kirr/go123/xsync.(*WorkGroup).Go.func1()
            /home/kirr/src/neo/src/lab.nexedi.com/kirr/go123/xsync/xsync.go:86 +0x68
    created by lab.nexedi.com/kirr/go123/xsync.(*WorkGroup).Go
            /home/kirr/src/neo/src/lab.nexedi.com/kirr/go123/xsync/xsync.go:83 +0x92
    >>> Change history by file:

    f<0000000000000002>:
                                    0 1 2 3 4 5 6 7
                                    a b c d e f g h
            @at0 (03e5a31e5e5ef6bb)
            @at1 (03e5a31e5e63fa77) 0

    ...

        @func
        def test_wcfs_crash_old_data():
            # start wcfs with ΔFtail/ΔBtail not covering that initial data.
            t = tDB(old_data=[{0:'a'}]); zf = t.zfile; at1 = t.head
            defer(t.close)

            f = t.open(zf)

            # ΔFtail coverage is currently (at1,at1]
            wl = t.openwatch()
            wl.watch(zf, at1, {})

            # wcfs is crashing on readPinWatcher -> ΔFtail.BlkRevAt with
            #   "at out of bounds: at: @at1,  (tail,head] = (@at1,@at1]
            # because BlkRevAt(at=tail) query was disallowed.
    >       f.assertBlk(0, 'a')          # [0] becomes tracked

Still also crashing in test_wcfs_watch_setup.

wcfs/wcfs_test.py

@@ -1822,19 +1822,28 @@ def test_wcfs_watch_2files():
 
 # ----------------------------------------
 
-# verify that wcfs does not panic with "no current transaction" when processing
-# invalidations if it needs to access ZODB during handleδZ.
+# verify that wcfs does not panic with "no current transaction" / "at out of
+# bounds" on read/invalidate/watch codepaths.
 @func
 def test_wcfs_crash_old_data():
     # start wcfs with ΔFtail/ΔBtail not covering initial data.
-    t = tDB(old_data=[{0:'a'}]); zf = t.zfile
+    t = tDB(old_data=[{0:'a'}]); zf = t.zfile; at1 = t.head
     defer(t.close)
 
     f = t.open(zf)
-    t.commit(zf, {1:'b1'})  # arbitrary commit to non-0 blk
-    f.assertBlk(0, 'a')     # [0] becomes tracked
 
-    # wcfs was crashing on processing further invalidation to blk 0 because
+    # ΔFtail coverage is currently (at1,at1]
+    wl = t.openwatch()
+    wl.watch(zf, at1, {})
+
+    # wcfs is crashing on readPinWatcher -> ΔFtail.BlkRevAt with
+    #   "at out of bounds: at: @at1,  (tail,head] = (@at1,@at1]
+    # because BlkRevAt(at=tail) query was disallowed.
+    f.assertBlk(0, 'a')          # [0] becomes tracked
+
+    at2 = t.commit(zf, {1:'b1'}) # arbitrary commit to non-0 blk
+
+    # wcfs was crashing on processing invalidation to blk 0 because
     # - ΔBtail.GetAt([0], head) returns valueExact=false, and so
     # - ΔFtail.BlkRevAt activates "access ZODB" codepath,
     # - but handleδZ was calling ΔFtail.BlkRevAt without properly putting zhead's transaction into ctx.