1. 11 Dec, 2023 1 commit
    • Kirill Smelkov's avatar
      Merge branch 'master' into y/nodefs-cancel · 9f9ad4a1
      Kirill Smelkov authored
      Notable patches:
      
      - 22d9c9dc "fuse: retain interface types"
      
        this removes one allocation from hot path.
      
      - 7b4f97c3 "fuse: add Options.Logger"
      
        this would allow to customize FUSE-related logging, if we would need.
      
      - 043296a8 "fuse: sync opcodes and capabilities"
      
        update from recent kernel.
      
      * master:
        fuse: parse socket control message using the standard library
        fs: support renameExchange in loopback for darwin
        fs: describe one more deadlock scenario
        fuse: extract macos and linux conflicted opcodes to platform specific builds
        fuse: move conflict opcodes into linux specific build
        fs: document known deadlocks
        fuse/test: avoid low-numbered fds for FUSE-backed files
        fuse: retain interface types
        fs: bridge: Add missing Lseek whence checks
        posixtest: run tests against normal filesystem
        fuse: document 0x8000 open flag
        fs: correct documentation on return errno codes
        fuse: sync opcodes and capabilities
        fuse: drop unused function
        fs: fix typo in comment
        Lazily init /dev/null fd
        fuse: add Options.Logger
      9f9ad4a1
  2. 20 Nov, 2023 1 commit
  3. 13 Nov, 2023 2 commits
  4. 10 Nov, 2023 2 commits
  5. 08 Nov, 2023 2 commits
    • Han-Wen Nienhuys's avatar
      fs: document known deadlocks · 1aa7b7b2
      Han-Wen Nienhuys authored
      Change-Id: I1e10f22dc13207277af45cbbc2ac6740c9d4e36e
      1aa7b7b2
    • Han-Wen Nienhuys's avatar
      fuse/test: avoid low-numbered fds for FUSE-backed files · 27a473d5
      Han-Wen Nienhuys authored
      As part of the fork/exec sequence, file descriptors that should be
      inherited are remapped with dup3() in the chld process. This causes
      implicit file close operations. If the file descriptors are backed by
      FUSE, this leads to FLUSH operations. If parallelism is limited with
      GOMAXPROCS, this can cause a deadlock, as there will not be threads
      left to service the FLUSH opcodes.
      
      Fixes #489
      
      Change-Id: I81bf4ab0624495aabb5bb9ec42a55c6f23340beb
      27a473d5
  6. 05 Nov, 2023 1 commit
    • Han-Wen Nienhuys's avatar
      fuse: retain interface types · 22d9c9dc
      Han-Wen Nienhuys authored
      This avoids one allocation under lock (calling sync.Pool.Put converts to
      interface{})
      
      Change-Id: Icfe155e34300dca398cea99b73bc5e10fd362ae3
      22d9c9dc
  7. 01 Oct, 2023 1 commit
    • Yasin Turan's avatar
      fs: bridge: Add missing Lseek whence checks · fc2c4d3d
      Yasin Turan authored
      The man page for lseek(2) says that the filesystem should
      return an ENXIO when the offset is greater than the file
      size on either SEEK_DATA or SEEK_HOLE or if the offset is
      set to the end of the file with SEEK_DATA (where there is
      an implicit hole). Adding these checks to the current Lseek
      implementation.
      
      Change-Id: I8db0913a9e37209e1681ffd11a9afb7f39810132
      fc2c4d3d
  8. 30 Sep, 2023 1 commit
  9. 18 Sep, 2023 2 commits
  10. 11 Sep, 2023 1 commit
    • Han-Wen Nienhuys's avatar
      fuse: sync opcodes and capabilities · 043296a8
      Han-Wen Nienhuys authored
      This adds opcodes and capabilities from this week's upstream kernel.
      
      As of commit 53db28933 ("fuse: extend init flags"), the init message
      has been extended to 104 bytes, with more flags added in a flags2
      field.
      
      Prepare for this by allowing _OP_INIT to have short reads.
      
      Tested on Fedora 38 (Linux 6.4.14) and Fedora 34 (Linux 5.11)
      
      Change-Id: I366ecda9e23f1a329134677075ee839674ff4c57
      043296a8
  11. 10 Sep, 2023 2 commits
  12. 09 Sep, 2023 1 commit
    • Brandon Duffany's avatar
      Lazily init /dev/null fd · b5c46276
      Brandon Duffany authored
      Fixes an issue that the init() func in splice.go panics if
      /dev is not yet mounted.
      
      This allows supporting go-based init binaries which set up all the
      filesystems on the machine, including both /dev as well as
      FUSE-mounted dirs.
      
      Change-Id: I74dab18effe4df410689ae7a2ef43670fcab7bd8
      b5c46276
  13. 28 Aug, 2023 1 commit
  14. 24 Jul, 2023 1 commit
    • Kirill Smelkov's avatar
      Merge branch 'master' into y/nodefs-cancel · aebdd447
      Kirill Smelkov authored
      Not notable patches - we only sync to pick up fixes for the tests and
      debug print adjustment to print flags with stable order.
      
      See nexedi/wendelin.core!16 (comment 188001)
      for context.
      
      * master:
        fs: return children in insertion order
        fs: move children map into separate struct
        fuse: TestDirectMount: ignore `Optional` field
        fuse: Don't print default if flags != 0
        fuse: Print debug flags in stable order
        fs: increase the limit for TestFileFdLeak
        fuse/test: Fix TestLargeDirRead
        .github: also test GOMAXPROCS=1
      aebdd447
  15. 22 Jun, 2023 2 commits
    • Han-Wen Nienhuys's avatar
      fs: return children in insertion order · 0b3e1fde
      Han-Wen Nienhuys authored
      Keep children in a slice of {name,*Inode} tuples, separately indexed
      by a string map.
      
      This ensures that we can return the children in insertion order. This
      fixes #476 and #391, and opens the road to NFS-compatible readdir
      cookies.
      
      Change-Id: I79b3483f4b12e67b63543da1294f725029a9139a
      0b3e1fde
    • Han-Wen Nienhuys's avatar
      fs: move children map into separate struct · 6eaf1ece
      Han-Wen Nienhuys authored
      Encapsulates the handling the children map, in preparation of changing
      the representation. It also reduces repetition for handling the
      parents field and changeCounter
      
      Change-Id: Ic741a376c6261dab606ed0705cdd130e70c8b69e
      6eaf1ece
  16. 21 Jun, 2023 1 commit
    • Jakob Unterwurzacher's avatar
      fuse: TestDirectMount: ignore `Optional` field · bb90a4c9
      Jakob Unterwurzacher authored
      As evidenced by the test failures that are reported, the
      `Optional` may change value between mounts. Ignore it
      in the comparison.
      
       --- FAIL: TestDirectMount (0.14s)
          mount_linux_test.go:173: DirectMount effective mount options mismatch:
              DirectMount: mountinfo.Info{ID:0, Parent:0, Major:0, Minor:0, Root:"", Mountpoint:"", Options:"rw,nosuid,nodev,relatime", Optional:"shared:417", FSType:"fuse./tmp/go-build3776165", Source:"a,b", VFSOptions:"rw,user_id=1000,group_id=1000,max_read=131072"}
              fusermount:  mountinfo.Info{ID:0, Parent:0, Major:0, Minor:0, Root:"", Mountpoint:"", Options:"rw,nosuid,nodev,relatime", Optional:"shared:373", FSType:"fuse./tmp/go-build3776165", Source:"a,b", VFSOptions:"rw,user_id=1000,group_id=1000,max_read=131072"}
      
      Fixes https://github.com/hanwen/go-fuse/issues/481
      
      Change-Id: Ie8105cf4bec7e7f461267217860cbf36e47ed329
      bb90a4c9
  17. 17 Jun, 2023 1 commit
  18. 16 Jun, 2023 2 commits
    • Kirill Smelkov's avatar
      fuse: Print debug flags in stable order · d5638ffd
      Kirill Smelkov authored
      This helps to compare fuse logs of e.g. different test runs. For example
      below are two INIT messages with the same flags, but formatted, before
      hereby patch, differently:
      
           rx 2: INIT n0 {7.37 Ra 131072 EXPORT_SUPPORT,SPLICE_READ,AUTO_INVAL_DATA,ASYNC_DIO,WRITEBACK_CACHE,BIG_WRITES,NO_OPEN_SUPPORT,PARALLEL_DIROPS,POSIX_ACL,EXPLICIT_INVAL_DATA,NO_OPENDIR_SUPPORT,ASYNC_READ,DONT_MASK,SPLICE_MOVE,IOCTL_DIR,READDIRPLUS,READDIRPLUS_AUTO,HANDLE_KILLPRIV,POSIX_LOCKS,ATOMIC_O_TRUNC,SPLICE_WRITE,FLOCK_LOCKS,ABORT_ERROR,MAX_PAGES,CACHE_SYMLINKS,0x70000000} "\x03\x00\x00\x00\x00\x00\x00\x00"... 48b
      
           rx 2: INIT n0 {7.37 Ra 131072 EXPORT_SUPPORT,SPLICE_READ,AUTO_INVAL_DATA,ASYNC_DIO,WRITEBACK_CACHE,BIG_WRITES,NO_OPEN_SUPPORT,PARALLEL_DIROPS,POSIX_ACL,EXPLICIT_INVAL_DATA,ASYNC_READ,DONT_MASK,SPLICE_MOVE,IOCTL_DIR,READDIRPLUS,READDIRPLUS_AUTO,HANDLE_KILLPRIV,NO_OPENDIR_SUPPORT,POSIX_LOCKS,ATOMIC_O_TRUNC,SPLICE_WRITE,FLOCK_LOCKS,ABORT_ERROR,MAX_PAGES,CACHE_SYMLINKS,0x70000000} "\x03\x00\x00\x00\x00\x00\x00\x00"... 48b
      
      which leads to corresponding lines to be present in the diff and
      requires human attention to verify that the messages are really the
      same.
      
      After this patch above INIT always formats as
      
           rx 2: INIT n0 {7.37 Ra 131072 ASYNC_READ,POSIX_LOCKS,ATOMIC_O_TRUNC,EXPORT_SUPPORT,BIG_WRITES,DONT_MASK,SPLICE_WRITE,SPLICE_MOVE,SPLICE_READ,FLOCK_LOCKS,IOCTL_DIR,AUTO_INVAL_DATA,READDIRPLUS,READDIRPLUS_AUTO,ASYNC_DIO,WRITEBACK_CACHE,NO_OPEN_SUPPORT,PARALLEL_DIROPS,HANDLE_KILLPRIV,POSIX_ACL,ABORT_ERROR,MAX_PAGES,CACHE_SYMLINKS,NO_OPENDIR_SUPPORT,EXPLICIT_INVAL_DATA,0x70000000} "\x03\x00\x00\x00\x00\x00\x00\x00"... 48b
      
      and the diff log does not unnecessarily distract attention to it.
      
      flagString is used for many messages, this fix should benefit all of
      them.
      
      Change-Id: Id4bdab9dd90552313ede41a3b9a6c337588e4b3e
      d5638ffd
    • Han-Wen Nienhuys's avatar
      fs: increase the limit for TestFileFdLeak · 9836d614
      Han-Wen Nienhuys authored
      The RELEASE opcode is asynchronous, so the next open() call may
      execute before the RELEASE for the preceding one has completed. In
      case of GOMAXPROCS=1, this happens often.
      
      Increase the limit to 15 to reduce the chance of this causing a test
      failure. The test runs 100 times, so this should still catch logic
      errors with deallocating file handles.
      
      Fixes #477.
      
      Change-Id: I12dbd96b2a6082e0f4a386d35983e6efcc596aef
      9836d614
  19. 14 Jun, 2023 3 commits
    • Kirill Smelkov's avatar
      fuse/test: Fix TestLargeDirRead · e5b78843
      Kirill Smelkov authored
      This test tries to create many files with random names and then asserts
      that readdir returns so many files that it created. But the test
      had a bug in that two random file names could be the same
      name but accounted twice. This leads to assertion in the end that the
      number of direntries read via readdir is not as expected.
      
      Here is how two random names could turn out to be the same:
      
      name 1: "file3" + "0" generated by randomLengthString
      name 2: "file30" + "" generated by randomLengthString
      
      -> Fix it by constructing random names guaranteed different.
      
      Fixes: https://github.com/hanwen/go-fuse/issues/472
      Change-Id: I776866bd728479da7324878d41dc71e504efd229
      e5b78843
    • Kirill Smelkov's avatar
      Merge branch 'master' into y/nodefs-cancel · eb4d413d
      Kirill Smelkov authored
      Notable patches:
      
      - 265a3926 "fuse: Increase MAX_KERNEL_WRITE to 1 MiB & enable CAP_MAX_PAGES"
      
        this should affect WCFS performance significantly because previously data was
        read from WCFS server in 128KB chunks and now it could be read in 2MB.
      
        For the reference: WCFS was setting MaxWrite=2MB from the beginning, and
        previously it was capped to 128KB by go-fuse. Now with go-fuse upgrade we
        should hopefully automatically get increase in performance.
      
      - 90b055af "fusermount: Fix option escaping"
      
        Levin's patch to allow ',' to be present in fsname, so that we can mount
        neo:// with multiple masters.
      
        See nexedi/wendelin.core!15 for context
        and details.
      
      * master: (45 commits)
        fs: kill subprocess before tearing down test
        fuse/test: Fix TestFopenKeepCache, take 2
        fs: simplify TestReadDirStress
        fuse: tweak Unmount doc comment
        .github: add Go 1.20
        fuse/mount_linux_test/DirectMount: Verify FsName \w comma/backslash works
        fusermount: Fix option escaping
        fuse: fix debug print for FsyncDir
        fuse: print GETATTR flags in input
        fs: document FileHandle argument for Getattr
        README.md: tweak & polish
        Drop "// " from LICENSE
        fs: fix NodeLookuper documentation
        .github: set user_allow_other in /etc/fuse.conf
        .github: drop Go 1.13 / 1.15, add Go 1.18/1.19
        fuse: MountDirect: always pass max_read
        tests: TestDirectMount: better coverage, clearer output
        fs: document Readdir determinism requirement
        fs: fold duplicate fuse.Context{} instantiations
        newunionfs: make readdir deterministic
        ...
      eb4d413d
    • Kirill Smelkov's avatar
      fixup! Y nodefs: Propagate context to File methods · 6e5a2cc2
      Kirill Smelkov authored
      Darwin build was failing:
      
          $ ./all.bash
          + go build ./...
          + GOOS=darwin
          + go build ./fuse/... ./fs/... ./example/loopback/...
          # github.com/hanwen/go-fuse/fuse/nodefs
          fuse/nodefs/files.go:99:9: cannot use &loopbackFile{…} (value of type *loopbackFile) as File value in return statement: *loopbackFile does not implement File (wrong type for method Allocate)
                          have Allocate(uint64, uint64, uint32) fuse.Status
                          want Allocate(uint64, uint64, uint32, *fuse.Context) (fuse.Status)
          fuse/nodefs/files_darwin.go:87:22: not enough arguments in call to f.GetAttr
                  have (*fuse.Attr)
                  want (*fuse.Attr, *fuse.Context)
      6e5a2cc2
  20. 12 Jun, 2023 2 commits
  21. 11 Jun, 2023 1 commit
    • Kirill Smelkov's avatar
      fuse/test: Fix TestFopenKeepCache, take 2 · d2138350
      Kirill Smelkov authored
      In 2019 TestOpenKeepCache was first disabled in ce2558b4 (fuse/test:
      disable TestFopenKeepCache) and then reenabled in 904ef0cc (fuse/test:
      Fix TestFopenKeepCache). The latter patch hoped to fix this test
      
          - by disabling CAP_AUTO_INVAL_DATA via ExplicitDataCacheControl. This
            should stop kernel from dropping data cache on mtime change;
          - by using the same size for before and after states. This avoid hitting
            cache being dropped when kernel sees file size being changed.
      
      In other words it took actions so that the data cache is not
      unexpectedly dropped by the kernel and the test does not fail with
      seeing different content after file reopen.
      
      However the test also wants to verify that updated data can be seen
      after cache invalidation and here it was not exactly correct: the test
      requested full control over the data cache via ExplicitDataCacheControl
      mount option, but was explicitly invalidating only file attributes
      via pathfs.EntryNotify _without_ invalidating the data cache.
      
      This way the test was actually working before Linux 5.2, because those
      older kernels never negotiate CAP_EXPLICIT_INVAL_DATA(*), but starting
      from Linux 5.2 CAP_EXPLICIT_INVAL_DATA started to be negotiated, and so
      the kernel was dropping only metadata without invalidating data cache on
      EntryNotify. This started to lead to the following test failures:
      
          --- FAIL: TestFopenKeepCache (0.13s)
              cache_test.go:153: ReadFile: got "before" after notify, want "afterX"
      
      @hanwen explains in https://github.com/hanwen/go-fuse/issues/473#issuecomment-1585617107
      
          It tries to invalidate the file content by issuing an ENTRY_NOTIFY. This
          leads to different mtime returned, but in the init phase, we don´t
          return AUTO_INVAL_DATA, because setupCacheTest explicitly disables it.
      
          === RUN   TestFopenKeepCache
          12:47:49.958227 callFusermount: executing ["/usr/bin/fusermount3" "/tmp/TestFopenKeepCache1094589261/001/mnt" "-o" "subtype=pathfs.pathInode,max_read=131072"]
          12:47:49.967664 rx 2: INIT n0 {7.38 Ra 131072 FLOCK_LOCKS,IOCTL_DIR,READDIRPLUS,READDIRPLUS_AUTO,PARALLEL_DIROPS,HANDLE_KILLPRIV,ASYNC_READ,BIG_WRITES,SPLICE_WRITE,SPLICE_MOVE,WRITEBACK_CACHE,NO_OPEN_SUPPORT,POSIX_ACL,POSIX_LOCKS,EXPORT_SUPPORT,AUTO_INVAL_DATA,ASYNC_DIO,MAX_PAGES,CACHE_SYMLINKS,NO_OPENDIR_SUPPORT,DONT_MASK,SPLICE_READ,ABORT_ERROR,EXPLICIT_INVAL_DATA,ATOMIC_O_TRUNC,0x70000000} "\a\x00\x00\x00\x00\x00\x00\x00"... 48b
          12:47:49.967708 tx 2:     OK, {7.28 Ra 131072 BIG_WRITES,NO_OPEN_SUPPORT,MAX_PAGES,EXPLICIT_INVAL_DATA,ASYNC_READ,READDIRPLUS,PARALLEL_DIROPS 0/0 Wr 131072 Tg 0 MaxPages 32}
      
      -> Let's fix this.
      
      As explained in 904ef0cc we do not want to reenable AUTO_INVAL_DATA
      because then the kernel will start dropping data cache on mtime change
      and the test will start to fail in its first part - where we want to
      assert that the data stays the same until data cache is invalidated.
      
      So let's change EntryNotify to FileNotify to explicitly invalidate data
      cache when we want the file to read with new data.
      
      Hopefully fixes: https://github.com/hanwen/go-fuse/issues/473
      
      (*) see git.kernel.org/linus/ad2ba64dd489 and github.com/hanwen/go-fuse/pull/273
      
      Change-Id: I4ff0f2bef61f0217601eee6d272f7e5563b272d2
      d2138350
  22. 09 Jun, 2023 2 commits
  23. 08 Jun, 2023 2 commits
  24. 07 Jun, 2023 1 commit
  25. 25 May, 2023 1 commit
  26. 18 May, 2023 2 commits
  27. 26 Apr, 2023 1 commit