When ReadDirPlus is enabled, the FUSE fs has to gather a lot of additional data for every file even if it won't be used (for example when the user is doing and 'ls' or a large glob). This can cause performance problems, so it should be possible to switch to the simple ReadDir behavior for file systems where this is a bottleneck.

Change-Id: I7f2e58fe6e728a408044ce789633423f6e527554

Change-Id: If4af8e09d1921242e36a415317d390154c5c2891

Change-Id: Icb2931139a6293abaaa95623a8e130dd6f8dab0b

There is a sync.Pool leak in server.go, which triggers in case of
multi-threaded loads.

Benchmarks:

env: Intel(R) Core(TM) i7-10700 CPU @ 2.90GHz (8core 16procs)
go test -bench="BenchmarkGoFuseRead" -benchmem

old:

2       10992 92501 ns/op 22671.71 MB/s 536981 B/op 196 allocs/op
4       19068 57959 ns/op 36183.44 MB/s 151877 B/op 170 allocs/op
8       27729 43796 ns/op 47884.39 MB/s 29068 B/op 162 allocs/op
16      28674 41403 ns/op 50651.65 MB/s 6704 B/op 160 allocs/op

fixed:

2       23667 46589 ns/op 45014.20 MB/s 5357 B/op 167 allocs/op
4       27753 42327 ns/op 49546.77 MB/s 4999 B/op 161 allocs/op
8       29648 40806 ns/op 51392.81 MB/s 4904 B/op 160 allocs/op
16      29389 40763 ns/op 51447.07 MB/s 4848 B/op 160 allocs/op

Change-Id: I062928203df5603ebca04c08c4d428d385f99c2c

The links pointed to a version of the docs from 2019.

Change-Id: I17a0739dbf23af015259ecb61493f304630548d7

Fixes rfjakob/gocryptfs#595 where we can't use gocryptfs -ro or gocryptfs -reverse on macOS because go-fuse tries to write to the read-only mountpoint during initial mount.

Fixes: #373
Change-Id: Ieadc0e074f4d803e88f9c21bb15b90eb9010afd8

OSXFUSE applies a 60-second timeout for file operations. This
is inconsistent with how FUSE works on Linux, where operations
last as long as the daemon is willing to let them run.

Change-Id: I6c042a3e785961a2c47c67e27d530bee5c50ce77

7.12 also allows usage of invalidation

Change-Id: I6e0480b7454187160f0b9f4b018386ba650048b7

These were deleted locally, but forgot to amend the commit.

Change-Id: I78c350aee09b6bb5d1a286514cae0c9a34d1396e

This library was created as a mechanism to overlay a writable checkout
for Git on top of Google's readonly SrcFS filesystem. This use case
has long disappeared, and the unionfs library appears unused
otherwise. Due to its reliance on timestamps and delays, it is very
susceptible to flakiness in tests.

Fixing this seems hard or maybe even impossible, so just remove the
library.

Change-Id: I6c594a724a0579b5c4c3aada76cd7013195fda94

Change-Id: I2b996ca3bfd1ea7e605e4dfd785cba2796340cd5

MKNOD returns an EntryOut that we can decode.

Before:

20:27:39.722507 rx 24914: MKNOD n1 {010644 (022), 0} ["pipe"] 5b
20:27:39.722546 tx 24914:     OK

After:

20: 32:39.156352 rx 38: MKNOD n1 {010644 (022), 0} ["pipe"] 5b
20: 32:39.156389 tx 38:     OK, {n3 g1 tE=1s tA=1s {M010644 SZ=0 L=1 1027:1027 B0*4096 i0:46906 A 1641756759.155157 M 1641756759.155157 C 1641756759.155157}}
Change-Id: If40e690219f20459466d7c8f9aa822c7186179dc

The readPool size was calculated from MaxWrite not taking
into account that the kernel fails all reads below 8 kiB.

Fix that by always using at least 8kiB.

Change-Id: I3bd9013be44b00bdc370bac00d00b4a106ec2d07

A later commit will allow setting MaxPages, so show it in the debug output.

Also print MaxReadahead, MaxWrite, TimeGran in decimal instead of hex -
these are neither bitfields nor addresses, so it seems clearer to have them in decimal.

Before:

22:03:50.020645 rx 2: INIT n0 {7.34 Ra 0x20000 PARALLEL_DIROPS,ASYNC_READ,EXPORT_SUPPORT,AUTO_INVAL_DATA,READDIRPLUS,NO_OPEN_SUPPORT,NO_OPENDIR_SUPPORT,EXPLICIT_INVAL_DATA,POSIX_LOCKS,DONT_MASK,SPLICE_READ,ASYNC_DIO,WRITEBACK_CACHE,HANDLE_KILLPRIV,CACHE_SYMLINKS,SPLICE_WRITE,SPLICE_MOVE,FLOCK_LOCKS,IOCTL_DIR,READDIRPLUS_AUTO,ABORT_ERROR,ATOMIC_O_TRUNC,BIG_WRITES,POSIX_ACL,MAX_PAGES,0x30000000}
22:03:50.020665 tx 2:     OK, {7.28 Ra 0x20000 AUTO_INVAL_DATA,READDIRPLUS,NO_OPEN_SUPPORT,PARALLEL_DIROPS,ASYNC_READ,MAX_PAGES,BIG_WRITES 0/0 Wr 0x2000 Tg 0x0}

After:

22: 02:51.374635 rx 2: INIT n0 {7.34 Ra 131072 POSIX_LOCKS,ATOMIC_O_TRUNC,DONT_MASK,FLOCK_LOCKS,AUTO_INVAL_DATA,READDIRPLUS_AUTO,PARALLEL_DIROPS,MAX_PAGES,ASYNC_READ,EXPORT_SUPPORT,BIG_WRITES,SPLICE_WRITE,SPLICE_READ,HANDLE_KILLPRIV,WRITEBACK_CACHE,CACHE_SYMLINKS,NO_OPENDIR_SUPPORT,EXPLICIT_INVAL_DATA,SPLICE_MOVE,IOCTL_DIR,READDIRPLUS,ASYNC_DIO,NO_OPEN_SUPPORT,POSIX_ACL,ABORT_ERROR,0x30000000}
22: 02:51.374656 tx 2:     OK, {7.28 Ra 131072 ASYNC_READ,BIG_WRITES,READDIRPLUS,NO_OPEN_SUPPORT,PARALLEL_DIROPS,AUTO_INVAL_DATA,MAX_PAGES 0/0 Wr 8192 Tg 0 MaxPages 2}
Change-Id: Iea02ad3270615d019413b3306e8a8e7282cd7c52

Change-Id: I7b8d1a6be288324021c846273aceb08eb9010e98

Since kernel 5.15, IOCTL returning ENOSYS is not allowed in overlayfs lowerdirs.
https://github.com/torvalds/linux/commit/72db82115d2bdfbfba8b15a92d91872cfe1b40c6

This commit fixes this issue by returning ENOTTY which is allowed since
https://github.com/torvalds/linux/commit/5b0a414d06c3ed2097e32ef7944a4abb644b89bd

This fixes the issue in 5.16 and later but not in 5.15.
To fully fix this, maybe we'll need to implement IOCTL.
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
Change-Id: Ib4a86a62e31980cd57d7153b78be82fa30190798

fusermount is (usually) from libfuse v2, and fusermount3 is from
libfuse v3, which is the actively-developed version.

Importantly, libfuse v3 added ExFAT to the list of filesystems
that users are allowed to mount on, and libfuse v2 did not.

The only ABI difference between fusermount and fusermount3 that
I am aware of is that fusermount3 dropped the allow_empty option.

Filesystems had to deal with that already, as fusermount may
also be from libfuse v3, depending on what the distro does.

This commit also adds two lines of debug output about which binary
we actually call.

Passes all.bash & the gocryptfs test suite.

Fixes https://github.com/hanwen/go-fuse/issues/394
Fixes https://github.com/rfjakob/gocryptfs/issues/626

Change-Id: Id4574fb9c8d2c812a524181a76616159256d551c

The workflow definition is adapted from gocryptfs,
that uses github actions for a month now.

Works ok as seen here:
https://github.com/rfjakob/go-fuse/actions/runs/1167431646

(except that some tests are flakey, but that's a
problem for another day).

Change-Id: Ib7aaa3b0a2f9671040fa599735aa47323d5c25d4

all.bash is broken for some time now (since the move to v2?).
Rewrite and simplify it using "./..." syntax.

And add the binaries that are generated to gitignore.

all.bash will be used in the upcomming patch for Github Actions CI.

Change-Id: Ia1b2e6bc60486b419408d4efd50bef4b8dd62bae

Unfortunately, I broke darwin with the last commit:

+ GOOS=darwin
+ GOARCH=amd64
+ go build -tags without_openssl -o /dev/null
../../../../pkg/mod/github.com/hanwen/go-fuse/v2@v2.1.1-0.20210825070001-74a933d6/fuse/server.go:122:5: undefined: parseFuseFd
../../../../pkg/mod/github.com/hanwen/go-fuse/v2@v2.1.1-0.20210825070001-74a933d6/fuse/server.go:887:5: undefined: parseFuseFd

Move parseFuseFd() to build on all archs to unbreak the build.

Change-Id: Ice173cc70a6a95765b56b444623b68ed92382052

libfuse introduced [1] a special `/dev/fd/N` syntax for the mountpoint:
It means that a privileged parent process:

 * Opened /dev/fuse
 * Called mount() on a real mountpoint directory
 * Inherited the fd to /dev/fuse to us
 * Informs us about the fd number via /dev/fd/N

This functionality is used to allow FUSE mounts inside containers
that have neither root permissions nor suid binaries [2], and
for the --drop_privileges flag of mount.fuse3 [4]

Tested with singularity and gocryptfs and actually works [3].

v2: Added doccomment for NewServer.
v3: Added specific error message on Server.Unmount().
v4: Moved mount details to package comment

[1] https://github.com/libfuse/libfuse/commit/64e11073b9347fcf9c6d1eea143763ba9e946f70
[2] https://github.com/rfjakob/gocryptfs/issues/590
[3] $ singularity run --fusemount "host:gocryptfs --extpass echo --extpass test /tmp/a /mnt" docker://ubuntu
    INFO:    Using cached SIF image
    Reading password from extpass program "echo", arguments: ["test"]
    Decrypting master key
    bash: /home/jakob/.cargo/env: No such file or directory
    bash: /home/jakob/.cargo/env: No such file or directory
    bash: /home/jakob/.cargo/env: No such file or directory
    Singularity> Filesystem mounted and ready.
[4] man mount.fuse3

Change-Id: Ibcc2464b0ef1e3d236207981b487fd9a7d94c910

Update the package comment to refer to the fs package
instead of nodefs and pathfs, which are deprecated.

Change-Id: I317e53152b63b982298a7de29120191488dcd362

This makes the function available to testing, which will
be used in the next commit.

Change-Id: I2df7aaf50748f209964da6f14764e81237d49027

We enable FUSE_CAP_ASYNC_READ per default, which means that
the kernel can (and does) submit multiple concurrent out-of-order
read requests to service userspace reads and kernel readahead.

For some backing storages, like Amazon Cloud Drive, out-of-order
reads are expensive.

gocryptfs has implemented a delay-based workaround with its
`-serialize_reads` flag for this case
(see https://github.com/rfjakob/gocryptfs/issues/92 for details).

Not enabling FUSE_CAP_ASYNC_READ makes the kernel do this for us,
as verified by adding debug output to gocryptfs, so expose it as
a mount flag in MountOptions.

Fixes: https://github.com/hanwen/go-fuse/issues/140
Graphs-at: https://github.com/hanwen/go-fuse/issues/395
Related: https://github.com/rfjakob/gocryptfs/issues/92
Change-Id: I10f947d71e1453989c4a9b66fbb0407f7163994f

The root directory always get nodeid 1, and the v2 api
originally had "nodeid == inode number".

But since 1d0096e5,
inode number and node id are independent, and we can allow
inode number 1.

Why does this matter? MacOS actually uses inode number 1 on ExFAT
mounts, triggering a panic on overlay filesystems like gocryptfs:

    panic: using reserved ID 1 for inode number

Fixes: https://github.com/rfjakob/gocryptfs/issues/585
Change-Id: I5e3476da65400d32efdb3dac1fe4901c64c00a89

Drops an obsolete entry pointing to go-fuse v1.0.0.

Change-Id: Iaf8e880d6f39e63b593a7929b8b4ff649c1d7807

The file type is not a bitmap, and looking at

	S_IFBLK                          = 0x6000
	S_IFCHR                          = 0x2000
	S_IFDIR                          = 0x4000
	S_IFIFO                          = 0x1000
	S_IFLNK                          = 0xa000
	S_IFREG                          = 0x8000
	S_IFSOCK                         = 0xc000

and confirmed by the added test, we used to classify
block devices and sockets as directories:

 --- FAIL: TestInodeIsDir (0.00s)
     inode_test.go:25: wrong result for case struct { mode uint32; dir bool }{mode:0x6000, dir:false}
     inode_test.go:25: wrong result for case struct { mode uint32; dir bool }{mode:0xc000, dir:false}

The check is fixed now and the test passes.

Change-Id: I6490992d1fecc8a6bea7c2c4b2f1bca765d03d4c

This

	GOMAXPROCS=1 go test -v -count=1 -run TestDeleteNotify ./fuse/test

deadlocked reliably, apparently in the chdir() from cmd.Start().

We can do without the chdir() by using an absolte path.
This avoids the issue.

Fixes https://github.com/hanwen/go-fuse/issues/261

Change-Id: Ia9c8f15819c125c5bf94b085fa4c5f2977a6789a

Maps do not free all memory when elements get deleted
( https://github.com/golang/go/issues/20135 ).

As a workaround, we recreate our two big maps (kernelNodeIds & stableAttrs)
when they have shrunk dramatically (100 x smaller).

Benchmarkung with loopback (go version go1.16.2 linux/amd64) shows:

Before this change:

Step               VmRSS (kiB)
-----              -----------
Fresh mount          4000
ls -R 500k files   271100
after drop_cache   336448
wait ~ 3 minutes   101588

After:

Step               VmRSS (kiB)
-----              -----------
Fresh mount          4012
ls -R 500k files   271100
after drop_cache    31528

Results for gocryptfs are similar.

Has survived xfstests via gocryptfs.

Fixes: https://github.com/rfjakob/gocryptfs/issues/569
Change-Id: Idcae1ab953270516735839a034d586717647b8db

inoMap & nodeidMap have been renamed a long time ago,
but this comment has been missed.

Update it for the new names.

Change-Id: I5e1d86f5e8ac3b3219b324da09d3f628b3f0fb80

If writing the INTERRUPT response returns ENOENT it means that the
referred request is no longer known by the kernel. This is a normal if
the referred request already has completed. We thus suppress the scary
warning unless debugging is enabled.

Fixes #375.

Change-Id: I3266dbd8ad53a42db9e0e746e4734f284746b76c

Enabling the logger shows this helpful warning

  warning: Inode.Path: n3 is orphaned, replacing segment with ".go-fuse.5577006791947779410/deleted"

for the problem reported at https://github.com/hanwen/go-fuse/issues/398 ,
and it generally makes sense to have complete logs for tests.

Enable the logger.

Change-Id: I4602f100a31c6e50bc48251a877b97ce3013ff24

Enables kernel ACL enforcement.

Change-Id: I7fb5f6f8b350ad430da18e0173df942b98da7c02

With

	CAP_POSIX_ACL           = (1 << 20)

the ACLs actually work.

Change-Id: Ic91e9f8d4196c811d4c566455b0e8c4a7871b105

Introduce the new struct inodeParents that wraps a map
and one special slot for the most recent parent. Unit tests
included.

Because the map is lazily initialized, we should save some
memory on the common single-parent case (= file with no hard links)
compared to before.

Benchmarking with gocryptfs shows no discernible change
in performance. fsstress testing with gocryptfs shows no issues.

TestStaleHardlinks the previous commit passes now.

Change-Id: I8d69093abc906addde751a9e70dbd78a3a61371a

This test shows a problem in selecting a random path
for a node when there is more than one.

The failure looks like this:

  18:41:50.796468 rx 136: LOOKUP n1 ["link0"] 6b
  18:41:50.796489 tx 136:     OK, {n2 g1 tE=0s tA=0s {M0100600 SZ=0 L=1 1026:1026 B0*4096 i0:269663 A 1616348510.793212 M 1616348510.793212 C 1616348510.795212}}
  18:41:50.796535 rx 138: OPEN n2 {O_RDONLY,0x8000}
  18:41:50.796557 tx 138:     2=no such file or directory, {Fh 0 }

The LOOKUP succeeds (because the file "link0" is there).
But the OPEN fails because go-fuse chooses another (stale) path.

I will try to make the behavoir more robust against changes
behind our back, but this patch only adds a test to show the
problem.

Change-Id: I39b31ba717ddaaad7dda6ecd86707c75cd25102e

Change-Id: Ide563371

Release isn't synchronized, so a quick [close,unlink] sequence may
still fail without this delay.

Change-Id: I64170fca

This is a more complete emulation of Windows FS semantics.

Change-Id: If9dec3ebeac808e1cc4f6d234290867c50e87a30

This avoids a panic.

Change-Id: I5c4cc0d875ecdbc3b07c2322b5a5a5e6fa493709