1. 08 Mar, 2019 4 commits
  2. 06 Mar, 2019 1 commit
  3. 24 Feb, 2019 2 commits
  4. 23 Feb, 2019 2 commits
    • Han-Wen Nienhuys's avatar
      fuse: support RENAME2 in RawFileSystem · fb9b7327
      Han-Wen Nienhuys authored
      In practice, this only triggers for RENAME_EXCHANGE, as the kernel
      issues the RENAME opcode in case of a normal rename or a
      RENAME_NOREPLACE.
      
      We have always advertised rename2 support (protocol v23), but it returned
      ENOSYS always. Instead, rename RenameIn => Rename1In, and add Flags to
      RenameIn.
      
      For raw filesystems that want to support rename2 should examine
      RenameIn.Flags and check for syscall.RENAME_XXX.
      
      For fuse/nodefs, we return ENOSYS in case of Rename2, since we lack
      API surface to propagate the flags.
      fb9b7327
    • Han-Wen Nienhuys's avatar
      afa3a28f
  5. 17 Feb, 2019 5 commits
  6. 14 Feb, 2019 1 commit
    • Kirill Smelkov's avatar
      fuse: Fix InodeRetrieveCache to retrieve data in chunks · 58dcd77a
      Kirill Smelkov authored
      It turned out that:
      
      1. Linux kernel cannot send data in chunks bigger than what was
         configured as MaxWrite on init, and
      
      2. if we ask for more, and that more, after capping to kernel write
         maxium, is more than configured MaxWrite, the kernel will simply drop
         NOTIFY_REPLY message. This, hopefully, will be fixed
      
      	https://marc.info/?l=linux-fsdevel&m=155000277921155&w=2
      
         but we cannot count on this fix to be present on older kernels, and
         even if it is there, we still won't get more than MaxWrite data.
      
      -> Rework InodeRetrieveCache to retrieve the data in chunks.
      
      NOTE1: we do not load users with this FUSE exchange details and instead
      of returning less content than requested, try to retrieve it all.
      
      NOTE2: without changes to fuse/server.go, added test will get stuck (see
      details in above link to linux-fsdevel patch).
      58dcd77a
  7. 13 Feb, 2019 7 commits
  8. 07 Feb, 2019 1 commit
    • Ed Schouten's avatar
      Don't crash when calling truncate(1) on a handle without a File. · 33711add
      Ed Schouten authored
      For all operations that can be applied on file descriptors, we have
      special treatment to permit the File to be nil. In that case, operations
      are called against the Node object instead.
      
      This check currently seems missing from SetAttr(), causing invocations
      of the truncate(1) command line tool to cause a crash:
      
              panic: runtime error: invalid memory address or nil pointer dereference
              [signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0xa30d0d]
      
              goroutine 12 [running]:
              github.com/hanwen/go-fuse/fuse/nodefs.(*rawBridge).SetAttr(0xc00013ecc0, 0xc0003a8000, 0xc0003ba0f0, 0xa27b98)
                      external/com_github_hanwen_go_fuse/fuse/nodefs/fsops.go:203 +0x58d
              github.com/hanwen/go-fuse/fuse.doSetattr(0xc0003a4000, 0xc0003ba000)
                      external/com_github_hanwen_go_fuse/fuse/opcode.go:171 +0x64
              github.com/hanwen/go-fuse/fuse.(*Server).handleRequest(0xc0003a4000, 0xc0003ba000, 0xc0003ba000)
                      external/com_github_hanwen_go_fuse/fuse/server.go:431 +0x26b
              github.com/hanwen/go-fuse/fuse.(*Server).loop(0xc0003a4000, 0xc00000b101)
                      external/com_github_hanwen_go_fuse/fuse/server.go:403 +0x18f
              created by github.com/hanwen/go-fuse/fuse.(*Server).readRequest
                      external/com_github_hanwen_go_fuse/fuse/server.go:291 +0x2d8
      33711add
  9. 05 Feb, 2019 1 commit
  10. 04 Feb, 2019 3 commits
  11. 01 Feb, 2019 2 commits
  12. 30 Jan, 2019 11 commits
    • Han-Wen Nienhuys's avatar
      unionfs: copy fuse.Attr before modifying return value of branch cache · 32b02fb6
      Han-Wen Nienhuys authored
      Fixes a race condition.
      32b02fb6
    • Han-Wen Nienhuys's avatar
    • Han-Wen Nienhuys's avatar
      unionfs: clarify test error message · 40675bc3
      Han-Wen Nienhuys authored
      40675bc3
    • Han-Wen Nienhuys's avatar
    • Jakob Unterwurzacher's avatar
      fuse: improve BATCH_FORGET debug log output · 8020c188
      Jakob Unterwurzacher authored
      Node numbers are now written as "i1234" everywhere except in the
      BATCH_FORGET output, fix that. Also show the transaction number
      so we can follow where each forget comes from.
      
      As a normal FORGET looks like this:
      
      	2019/01/27 14:38:07 rx 1488: FORGET i822 {Nlookup=1}
      
      with the new output, "grep FORGET i1234 " will catch both single forgets
      and batch forgets.
      
      Before:
      
      	2019/01/20 18:15:31 rx 1372: BATCH_FORGET i0 {Count=3}  48b
      	2019/01/20 18:15:31 doBatchForget: forgetting 1 of 3: NodeId: 40, Nlookup: 1
      	2019/01/20 18:15:31 doBatchForget: forgetting 2 of 3: NodeId: 41, Nlookup: 1
      	2019/01/20 18:15:31 doBatchForget: forgetting 3 of 3: NodeId: 42, Nlookup: 1
      
      After:
      
      	2019/01/27 14:48:35 rx 2982: BATCH_FORGET i0 {Count=2}  32b
      	2019/01/27 14:48:35 doBatchForget: rx 2982 1/2: FORGET i804 {Nlookup=1}
      	2019/01/27 14:48:35 doBatchForget: rx 2982 2/2: FORGET i803 {Nlookup=1}
      8020c188
    • Kirill Smelkov's avatar
      nodefs: Don't crash on handleless WithFlags opens · 402331d4
      Kirill Smelkov authored
      Since https://git.kernel.org/linus/7678ac5061 FUSE filesystems were allowed to
      return ENOSYS from open and, open seeing this, the kernel will not send any
      open request anymore and just use Fh=0 for all opened files. This is handy for
      filesystems that don't have any per-opened-file state.
      
      However if there is at least one node for which opened files should have
      their own state, it is not correct to return ENOSYS from open _ever_, as that
      would prevent the kernel to call open on all nodes - in particular on the node
      where open needs to create a state associated with the file handle(*).
      
      It is already explicitly documented that Node.Open can return File=nil
      in which case filesystem operations like Read and Write will be called
      on the node directly. This way a filesystem could try to simulate an
      Open that was returning ENOSYS with Open that returns File=nil. However
      it is not the same as ENOSYS open also prevents the kernel from dropping
      the file cache:
      
      https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/fuse/file.c?id=v5.0-rc3-241-g7c2614bf7a1f#n131
      
      From this point of view Opens that were returning ENOSYS should be
      replaced with Open that returns WithFlags{File=nil, flags=FOPEN_KEEP_CACHE}.
      
      Unfortunately if one tries to do that, go-fuse segfaults as the test
      included in this patch demonstrates:
      
      	13:36:53.238706 rx 13: LOOKUP i1 ["world.txt"] 10b
      	13:36:53.238745 tx 13:     OK, {i4 g3 tE=1s tA=1s {M0100644 SZ=5 L=1 1000:1000 B0*0 i0:4 A 0.000000 M 0.000000 C 0.000000}}
      	13:36:53.238780 rx 14: OPEN i4 {O_RDONLY,0x8000}
      	13:36:53.238791 tx 14:     OK, {Fh 2 CACHE}				<-- NOTE Fh != 0
      	13:36:53.238706 rx 12: RELEASE i3 {Fh 0 NONBLOCK,0x8000  L0}
      	13:36:53.238804 tx 12:     OK
      	13:36:53.238823 rx 15: READ i4 {Fh 2 [0 +4096)  L 0 NONBLOCK,0x8000}
      	13:36:53.238830 tx 15:     OK,  5b data "world"
      	13:36:53.238846 rx 16: GETATTR i4 {Fh 2}
      	13:36:53.238865 tx 16:     OK, {tA=1s {M0100644 SZ=5 L=1 1000:1000 B0*0 i0:4 A 0.000000 M 0.000000 C 0.000000}}
      	13:36:53.238879 rx 17: FLUSH i4 {Fh 2}
      	panic: runtime error: invalid memory address or nil pointer dereference
      	[signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0x539f9f]
      
      	goroutine 6 [running]:
      	github.com/hanwen/go-fuse/fuse/nodefs.(*rawBridge).Flush(0xc000010960, 0xc000104180, 0x0)
      	        /home/kirr/src/neo/src/github.com/hanwen/go-fuse/fuse/nodefs/fsops.go:490 +0x6f
      	github.com/hanwen/go-fuse/fuse.doFlush(0xc0000de000, 0xc000104000)
      	        /home/kirr/src/neo/src/github.com/hanwen/go-fuse/fuse/opcode.go:373 +0x42
      	github.com/hanwen/go-fuse/fuse.(*Server).handleRequest(0xc0000de000, 0xc000104000, 0xc000104000)
      	        /home/kirr/src/neo/src/github.com/hanwen/go-fuse/fuse/server.go:431 +0x26b
      	github.com/hanwen/go-fuse/fuse.(*Server).loop(0xc0000de000, 0x0)
      	        /home/kirr/src/neo/src/github.com/hanwen/go-fuse/fuse/server.go:403 +0x18f
      	github.com/hanwen/go-fuse/fuse.(*Server).Serve(0xc0000de000)
      	        /home/kirr/src/neo/src/github.com/hanwen/go-fuse/fuse/server.go:331 +0x6d
      	created by github.com/hanwen/go-fuse/fuse/test.TestNoFile
      	        /home/kirr/src/neo/src/github.com/hanwen/go-fuse/fuse/test/nofile_test.go:73 +0x442
      
      Fix it by teaching registerFileHandle not to register any handle at all
      and return Fh=0, if the inner file (file itself, or the file that was wrapped
      with WithFlags) is nil.
      
      After the patch the trace for world.txt open/read (that is opened with
      WithFlags{File=nil, Flags=FOPEN_KEEP_CACHE} is as follows.
      
      	14:59:31.714048 rx 13: LOOKUP i1 ["world.txt"] 10b
      	14:59:31.714062 tx 13:     OK, {i4 g3 tE=1s tA=1s {M0100644 SZ=5 L=1 1000:1000 B0*0 i0:4 A 0.000000 M 0.000000 C 0.000000}}
      	14:59:31.714081 rx 14: OPEN i4 {O_RDONLY,0x8000}
      	14:59:31.714091 tx 14:     OK, {Fh 0 CACHE}				<-- NOTE Fh = 0
      	14:59:31.714123 rx 15: READ i4 {Fh 0 [0 +4096)  L 0 NONBLOCK,0x8000}
      	14:59:31.714132 tx 15:     OK,  5b data "world"
      	14:59:31.714150 rx 16: GETATTR i4 {Fh 0}
      	14:59:31.714159 tx 16:     OK, {tA=1s {M0100644 SZ=5 L=1 1000:1000 B0*0 i0:4 A 0.000000 M 0.000000 C 0.000000}}
      	14:59:31.714181 rx 17: FLUSH i4 {Fh 0}
      	14:59:31.714186 tx 17:     OK
      	14:59:31.714202 rx 18: RELEASE i4 {Fh 0 NONBLOCK,0x8000  L0}
      	14:59:31.714208 tx 18:     OK
      
      (*) my particular case is filesystem where many nodes are just data, but
      additionally there are files that behave like sockets - for every client
      who opens such file the filesystem establishes separate bidirectional
      channel for control exchange via that stream:
      
      https://lab.nexedi.com/kirr/wendelin.core/blob/a50da567fd/wcfs/misc.go#L205
      402331d4
    • Han-Wen Nienhuys's avatar
      unionfs: fix capitalization of TTL · f45e18d6
      Han-Wen Nienhuys authored
      f45e18d6
    • Han-Wen Nienhuys's avatar
      fuse/nodefs: protect mutable memnode fields with lock · 7f7894d2
      Han-Wen Nienhuys authored
      This fixes race conditions found by the race detector.
      7f7894d2
    • Han-Wen Nienhuys's avatar
      unionfs: update in-memory dircache as the last step. · e6fa1a12
      Han-Wen Nienhuys authored
      If we call AddEntry/RemoveEntry before the update to the deletion list
      in storage, a poorly timed cache refill that happens just before the
      storage update will result in storing the old state for as long as the
      cache's TTL.
      e6fa1a12
    • Han-Wen Nienhuys's avatar
      e00a5f2a
    • Han-Wen Nienhuys's avatar
      aed5935e