• Kirill Smelkov's avatar
    wcfs: Initial implementation of basic filesystem · 58e2a88c
    Kirill Smelkov authored
    Provide filesystem view of in-ZODB ZBigFiles, but do not implement support for
    invalidations nor isolation protocol yet. In particular, because ZODB
    invalidations are not yet handled, the filesystem does not update its data in
    accordance with ZODB updates, and instead provides stale data view that
    corresponds to the state of ZODB at the time when wcfs was mounted.
    
    The main parts of this patch are:
    
    - wcfs/wcfs.go is filesystem implementation itself together with overview.
    - wcfs/__init__.py is python wrapper to spawn and interoperate with that filesystem.
    - wcfs/wcfs_test.py is tests.
    
    Some preliminary history:
    
    fe7efb94    X start of wcfs
    878b2787    X draft loading
    d58c71e8    X don't overalign end by 1 blksize if end is already aligned
    29c9f13d    X readBlk: Fix thinko in already case
    59552328    X wcfs: Care to disable OS polling on us
    c00d94c7    X workaround lack of exception chaining on Python2 with xdefer
    0398e23d    X bytearray turned out to be copying data
    7a837040    X print wcfs.py py-level traceback on SIGBUS (e.g. wcfs.go aborting due to bug/panic)
    661b871f    X make sure tests don't get stuck even if wcfs gets killed -9 ...
    2c043d29    X More effort to unmount failed wcfs.go
    1ccc4478    X Use `with gil` + regular py code instead of PyGILState_Ensure/PyGILState_Release/PyRun_SimpleString
    5dc9c791    X wcfs: Kill xdefer
    91e9eba8    X wcfs: test: Register tFile to tDB early
    a7138fef    X wcfs: mkdir /tmp/wcfs with sticky bit
    1eec76d0    X wcfs: try to set sticky for /tmp/wcfs even if the directory already exists
    c2c35851    X wcfs: tests: Factor-out waiting for a general condition to become true into waitfor
    78f36993    X wcfs: test: Fix thinko in getting /sys/fs/fuse/connection/<X> for wcfs
    bc9eb16f    X wcfs: tests: Don't use testmntpt everywhere
    6dec74e7    X wcfs: tests: Split tDB into -> tDB + tWCFS
    3a6bd764    X wcfs: tests: Run `fusermount -u` the second time if we had to kill wcfs
    112720f3    X wcfs: tests: Print which files are still opened on wcfs if `fusermount -u` fails
    bb40185b    X wcfs: Take $WENDELIN_CORE_WCFS_OPTIONS into account not only from under join
    03a9ef33    X wcfs: Remove credentials from zurl when computing wcfs mountpoint
    68ee5bdc    X wcfs: lsof tweaks
    21671879    X wcfs: Teach entrypoint frontend to handle subcommands: serve, status, stop
    b0642b80    X wcfs: Switch mountpoints from /tmp/wcfs/* to /dev/shm/*
    b0ca031f    X wcfs: Teach join/serve to start successfully even after unclean wcfs shutdown
    5bfa8cf8    X wcfs: Add start to spawn a Server that can be later stopped  (draft)
    5fcec261    X wcfs: Run fusermount and friends with /bin:/usr/bin always on path
    669d7a20    fixup! X wcfs: Run fusermount and friends with /bin:/usr/bin always on path
    6b22f8c4    X wcfs: Teach start to start successfully even after unclean wcfs shutdown
    15389db0    X wcfs: Tune _fuse_unmount to include `fusermount -u` error message into raised exception
    153c002a    X wcfs: _fuse_unmount: Try first `kill -TERM` before `kill -QUIT` wcfs
    3244f3a6    X wcfs: lsof +D misbehaves - don't use it
    a126e709    X wcfs: Put client log into its own logger
    ac303d1e    X wcfs: tests: -v  ->  show only wcfs.py logs verbosely
    d671a9e9    X wcfs: Give more time to stop wcfs server
    58e2a88c
wcfs.go 24.6 KB