- 25 Apr, 2019 7 commits
-
-
Kirill Smelkov authored
We have many calls to ioutil.WriteFile and ioutil.ReadFile + checking for error. Move those calls into utility functions which call t.Fatal if they see any error. No need to use additional error prefix as both ioutil.WriteFile and ReadFile produce os.PathError which always has opertion and path on which it was performed.
-
Kirill Smelkov authored
Add nodefs.Mount - utility to mount root over mountpoint with given options. We already had nodefs.MountRoot, but that was taking only nodefs.Options and there was no way to pass fuse.MountOptions in. The new utility accepts both fuse.MountOptions and nodefs.Options, each covering their level. This should be generally useful(*), as well as it will be used in a next patch in TestFopenKeepCache where fuse.MountOptions.PreciseDataCacheControl will need to be used. (*) see e.g. https://lab.nexedi.com/kirr/wendelin.core/blob/8f497094/wcfs/misc.go#L258 as example that users unroll their Mount versions to be able to pass in fuse.MountOptions.
-
Kirill Smelkov authored
CAP_AUTO_INVAL_DATA is capability of kernel, but from the point of view of a filesystem it is not a capability, but a behaviour request: if set, it requests to kernel - filesystem client - to perform data cache invalidations based on heuristics. Current heuristic is to drop data cache for a file if kernel sees file's mtime being changed: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/fuse/inode.c?id=v5.0-0-g1c163f4c7b3f#n238 https://git.kernel.org/linus/eed2179efe However this could be unwanted behaviour if filesystem is careful to explicitly invalidate local file cache: despite filesystem attempts to preciously keep data cache, the whole cache of the file is unnecessarily dropped. -> To fix add a new mount options for filesystems to indicate they are careful with respect to data cache invalidations and thus should be fully responsible for invalidating data cache. Teach go-fuse to not send CAP_AUTO_INVAL_DATA in this mode to kernel on FUSE handshake. Note, as of upcoming Linux 5.1 (estimated to be released mid 2019), FUSE client in kernel still automatically and unconditionally drops whole data cache of a file if its sees size change. Kernel and go-fuse patches to fix that are here: https://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse.git/commit/?id=ad2ba64dd489 https://github.com/hanwen/go-fuse/pull/273 The kernel patch is likely to enter mainline kernel when 5.2 merge window opens. We'll remove XXX in a follow-up go-fuse patch that adds support for CAP_EXPLICIT_INVAL_DATA.
-
Kirill Smelkov authored
FOPEN_STREAM, together with FOPEN_NONSEEKABLE must be used on stream-like file handles that provide both read and write to avoid hitting deadlock in the kernel. Please see the following kernel patch for details on how the deadlock can happen: git.kernel.org/linus/10dce8af3422 Adding FOPEN_STREAM to kernel FUSE is in fuse.git#for-next now and is likely to enter mainline kernel when 5.2 merge window opens way or another: https://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse.git/commit/?id=bbd84f33652 https://lore.kernel.org/linux-fsdevel/CAHk-=whQQdoQsgEx1vO7OkfPDcV5hurPnMRLgzfXAPN63n5Sbg@mail.gmail.com/ https://lore.kernel.org/linux-fsdevel/CAHk-=wjEOyba5As1PEMk6RitNVOJH9oJ_Jbg4y=5B0fcX1iKGw@mail.gmail.com/ https://lore.kernel.org/linux-fsdevel/CAHk-=wgh234SyBG810=vB360PCzVkAhQRqGg8aFdATZd+daCFw@mail.gmail.com/ https://lore.kernel.org/linux-fsdevel/20190424183012.GB3798@deco.navytux.spb.ru/ Here is example for FOPEN_STREAM usage: https://lab.nexedi.com/kirr/wendelin.core/blob/7783ecf4/wcfs/misc.go#L327-335 https://lab.nexedi.com/kirr/wendelin.core/blob/7783ecf4/wcfs/misc.go#L276-428
-
Han-Wen Nienhuys authored
-
Han-Wen Nienhuys authored
-
Han-Wen Nienhuys authored
-
- 23 Apr, 2019 1 commit
-
-
Kirill Smelkov authored
When kernel sends WRITE request it sends with it file handle, offset, size etc. We were not printing all this. Compare e.g. debug output sample for TestUtimesNano before and after the patch. before: rx 8: CREATE i1 {0100600 [CREAT,TRUNC,WRONLY,0x8000] (00)} ["hello.txt"] 10b tx 8: OK, {i3 g2 {M0100600 SZ=0 L=1 1000:1000 B0*4096 i0:733462 A 1556038355.426630 M 1556038355.426630 C 1556038355.426630} &{2 0 0}} rx 9: GETXATTR i3 {sz 0} ["security.capability"] 20b tx 9: 61=no data available rx 10: WRITE i3 3b <-- NOTE tx 10: OK after: rx 8: CREATE i1 {0100600 [WRONLY,CREAT,TRUNC,0x8000] (00)} ["hello.txt"] 10b tx 8: OK, {i3 g2 {M0100600 SZ=0 L=1 1000:1000 B0*4096 i0:736300 A 1556038379.359197 M 1556038379.359197 C 1556038379.359197} &{2 0 0}} rx 9: GETXATTR i3 {sz 0} ["security.capability"] 20b tx 9: 61=no data available rx 10: WRITE i3 {Fh 2 [0 +3) L 0 WRONLY,NONBLOCK,0x8000} 3b <-- NOTE tx 10: OK
-
- 17 Apr, 2019 4 commits
-
-
Han-Wen Nienhuys authored
The new naming makes clear that this is the One True API for Go-FUSE
-
Han-Wen Nienhuys authored
-
Han-Wen Nienhuys authored
-
Han-Wen Nienhuys authored
-
- 16 Apr, 2019 2 commits
-
-
Jakob Unterwurzacher authored
There is a hang that appears when enabling CAP_PARALLEL_DIROPS on Linux 4.15.0: https://github.com/hanwen/go-fuse/issues/281 The hang was originally triggered by gvfs-udisks2-volume-monitor. This test emulates what gvfs-udisks2-volume-monitor does. On 4.15.0 kernels, the test will get stuck, and after 120 seconds you get a kernel backtrace like this: [ 1813.463679] INFO: task nodefs.test:2357 blocked for more than 120 seconds. [ 1813.463685] Not tainted 4.15.0-45-generic #48~16.04.1-Ubuntu [ 1813.463687] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 1813.463689] nodefs.test D 0 2357 2311 0x00000004 [ 1813.463691] Call Trace: [ 1813.463709] __schedule+0x3d6/0x8b0 [ 1813.463712] schedule+0x36/0x80 [ 1813.463714] schedule_preempt_disabled+0xe/0x10 [ 1813.463716] __mutex_lock.isra.2+0x2ae/0x4e0 [ 1813.463720] ? ___slab_alloc+0x223/0x4e0 [ 1813.463722] ? _cond_resched+0x1a/0x50 [ 1813.463724] __mutex_lock_slowpath+0x13/0x20 [ 1813.463725] ? __mutex_lock_slowpath+0x13/0x20 [ 1813.463727] mutex_lock+0x2f/0x40 [ 1813.463729] fuse_lock_inode+0x2a/0x30 [ 1813.463732] fuse_lookup+0x31/0x140 [ 1813.463735] ? d_alloc_parallel+0xc1/0x4c0 [ 1813.463738] fuse_atomic_open+0x6d/0xf0 [ 1813.463740] path_openat+0xc5d/0x13f0 [ 1813.463744] do_filp_open+0x99/0x110 [ 1813.463747] ? __check_object_size+0xfc/0x1a0 [ 1813.463749] ? __alloc_fd+0x46/0x170 [ 1813.463752] do_sys_open+0x12d/0x290 [ 1813.463754] ? do_sys_open+0x12d/0x290 [ 1813.463756] SyS_openat+0x14/0x20 [ 1813.463759] do_syscall_64+0x73/0x130 [ 1813.463762] entry_SYSCALL_64_after_hwframe+0x3d/0xa2
-
Jakob Unterwurzacher authored
The old code searched for the first non-null byte from the end of the slice. This assumes that all bytes after the name are initialized to zero, which does not hold true on Linux 4.15. Instead, search for the first null byte from the start of the slice, which is guaranteed by man(3) readdir. Fixes https://github.com/hanwen/go-fuse/issues/287
-
- 09 Apr, 2019 5 commits
-
-
Han-Wen Nienhuys authored
-
Han-Wen Nienhuys authored
-
Han-Wen Nienhuys authored
-
Han-Wen Nienhuys authored
-
Han-Wen Nienhuys authored
This reduces the debug output from 25k lines to ~1k
-
- 08 Apr, 2019 8 commits
-
-
Kirill Smelkov authored
-
Han-Wen Nienhuys authored
-
Han-Wen Nienhuys authored
-
Han-Wen Nienhuys authored
-
Han-Wen Nienhuys authored
-
Han-Wen Nienhuys authored
-
Han-Wen Nienhuys authored
-
Han-Wen Nienhuys authored
-
- 07 Apr, 2019 3 commits
-
-
Han-Wen Nienhuys authored
Appeases the race detector.
-
Jakob Unterwurzacher authored
This currently fails on Linux 5.0 and may be related to https://github.com/hanwen/go-fuse/issues/287 . 1 jakob@brikett:~/go/src/github.com/hanwen/go-fuse/nodefs$ go test 21:42:47.356529 writer: Write/Writev failed, err: 2=no such file or directory. opcode: RELEASE 21:42:47.598309 writer: Write/Writev failed, err: 22=invalid argument. opcode: READDIRPLUS 21:42:47.604424 writer: Write/Writev failed, err: 22=invalid argument. opcode: READDIRPLUS 21:42:47.606073 writer: Write/Writev failed, err: 22=invalid argument. opcode: READDIRPLUS --- FAIL: TestReadDirStress (0.36s) simple_test.go:270: goroutine 2 iteration 5: readdirent: input/output error simple_test.go:270: goroutine 1 iteration 9: readdirent: input/output error simple_test.go:270: goroutine 3 iteration 10: readdirent: input/output error simple_test.go:43: /usr/bin/fusermount: entry for /tmp/TestReadDirStress639795934/mnt not found in /etc/mtab (code exit status 1) FAIL exit status 1 FAIL github.com/hanwen/go-fuse/nodefs 0.994s
-
Han-Wen Nienhuys authored
Access file descriptors under lock, and set to -1 on close. This avoids confusing errors if Close() is doubly called
-
- 06 Apr, 2019 10 commits
-
-
Han-Wen Nienhuys authored
-
Han-Wen Nienhuys authored
-
Han-Wen Nienhuys authored
-
Han-Wen Nienhuys authored
-
Han-Wen Nienhuys authored
This is based on the new nodefs API, and tests the new API for writing a full-fledged r/w filesystem.
-
Han-Wen Nienhuys authored
-
Han-Wen Nienhuys authored
-
Han-Wen Nienhuys authored
-
Han-Wen Nienhuys authored
-
Han-Wen Nienhuys authored
-