Skip to content

G
go-fuse
Commit 43d5f40d authored by Kirill Smelkov's avatar Kirill Smelkov
Browse files
Options

Merge branch 'y/nodefs-cancel' into t 

* y/nodefs-cancel: (60 commits)
  Make ReadDirPlus configurable
  all.bash: use /bin/bash explicitly
  fuse: Add release flag constants
  Fix sync.Pool leak
  README: make godoc links point to v2 instead of ancient v1.0.0
  Ports commit fee50bf0 to poll_darwin.go in order to allow read-only mounts on Darwin as well.
  Make OSXFUSE handle timeouts the same way as Linux
  Update to fuse 7.12
  example: remove unionfs examples
  unionfs: remove library
  posixtest: provide more detail for failing ReadDir test
  fuse: decode MKNOD result in debug output
  fuse: fix mounting with MaxWrite < 8kiB
  fuse: add MaxPages to INIT debug output
  fs: update oldParent.children for exchange rename
  Fix IOCTL to return the error allowed by overlayfs
  fuse: prefer fusermount3 over fusermount; add debug output
  Replace defunct travis ci with github actions
  Rewrite defunct all.bash
  fuse: move parseFuseFd() to unbreak darwin build
  ...
parents 51ae026e 643585d7
Hide whitespace changes
Inline Side-by-side
Showing with 2038 additions and 4634 deletions
.github/workflows/ci.yml 0 → 100644
View file @ 43d5f40d
name: CI
on:
push:
pull_request:
schedule:
- cron: '0 12 * * *' # Every day noon UTC
jobs:
build:
strategy:
matrix:
go:
- "1.13.x" # Ubuntu 20.04 LTS "focal"
- "1.15.x" # Debian 11 "Bullseye"
- "1.16.x" # Golang upstream stable
- "1.17.x" # Golang upstream stable
# Don't cancel everything when one Go version fails
fail-fast: false
runs-on: ubuntu-latest
steps:
- name: Install Go ${{ matrix.go }}
uses: actions/setup-go@v2
with:
go-version: ${{ matrix.go }}
- uses: actions/checkout@v2
with:
fetch-depth: 0 # Make "git describe" work
# CI platform specific setup steps happen here
- run: sudo apt-get install -qq fuse3 libssl-dev libfuse-dev
# Actual test steps are in all.bash
- run: ./all.bash
.gitignore
View file @ 43d5f40d
......@@ -14,3 +14,6 @@ example/bulkstat/bulkstat
example/zipfs/zipfs
# test binaries
*.test
benchmark/bulkstat.bin
benchmark/bulkstat/main
benchmark/cstatfs
.travis.yml deleted 100644 → 0
View file @ 51ae026e
sudo: required
# Ubuntu 18.04 "Bionic", https://docs.travis-ci.com/user/reference/bionic/
# Kernel 5.0.0-1026-gcp
dist: bionic
language: go
go_import_path: github.com/hanwen/go-fuse
go:
- 1.10.x
- 1.11.x
- 1.12.x
- 1.13.x
- master
matrix:
fast_finish: true
allow_failures:
- go: master
before_install:
- sudo apt-get install -qq pkg-config fuse
- sudo modprobe fuse
- sudo chmod 666 /dev/fuse
- sudo chown root:$USER /etc/fuse.conf
install:
- go get -t ./...
- go get -t -race ./...
# Travis CI has a no-output-timeout of 10 minutes.
# Set "go test -timeout" lower so we get proper backtraces
# on a hung test.
# The tests sometimes hang in a way that "go test -timeout"
# does not work anymore. Use the external "timeout" command
# as backup, triggering 1 minute later.
script:
- set -e # fail fast
- timeout -s QUIT -k 10s 90s go test -failfast -timeout 1m -p 1 -v ./fs
- timeout -s QUIT -k 10s 6m go test -failfast -timeout 5m -p 1 -v ./...
- set +e # restore
AUTHORS
View file @ 43d5f40d
......@@ -16,5 +16,6 @@ Patrick Crosby <pcrosby@gmail.com>
Paul Jolly <paul@myitcv.org.uk>
Paul Warren <paul.warren@emc.com>
Shayan Pooya <shayan@arista.com>
Tommy Lindgren <tommy.lindgren@gmail.com>
Valient Gough <vgough@pobox.com>
Yongwoo Park <nnnlife@gmail.com>
README.md
View file @ 43d5f40d
# GO-FUSE
[![Build Status](https://travis-ci.org/hanwen/go-fuse.svg?branch=master)](https://travis-ci.org/hanwen/go-fuse)
[![GoDoc](https://godoc.org/github.com/hanwen/go-fuse?status.svg)](https://godoc.org/github.com/hanwen/go-fuse)
[![CI](https://github.com/hanwen/go-fuse/actions/workflows/ci.yml/badge.svg)](https://github.com/hanwen/go-fuse/actions/workflows/ci.yml)
[![GoDoc](https://godoc.org/github.com/hanwen/go-fuse?status.svg)](https://godoc.org/github.com/hanwen/go-fuse/v2)
Go native bindings for the FUSE kernel module.
You should import and use
[github.com/hanwen/go-fuse/fs](https://godoc.org/github.com/hanwen/go-fuse/fs)
[github.com/hanwen/go-fuse/v2/fs](https://godoc.org/github.com/hanwen/go-fuse/v2/fs)
library. It follows the wire protocol closely, but provides
convenient abstractions for building both node and path based file
systems
Older, deprecated APIs are available at
[github.com/hanwen/go-fuse/fuse/pathfs](https://godoc.org/github.com/hanwen/go-fuse/fuse/pathfs)
[github.com/hanwen/go-fuse/fuse/pathfs](https://godoc.org/github.com/hanwen/go-fuse/v2/fuse/pathfs)
and
[github.com/hanwen/go-fuse/fuse/nodefs](https://godoc.org/github.com/hanwen/go-fuse/fuse/nodefs).
[github.com/hanwen/go-fuse/fuse/nodefs](https://godoc.org/github.com/hanwen/go-fuse/v2/fuse/nodefs).
## Comparison with other FUSE libraries
The FUSE library gained a new, cleaned-up API during a rewrite
completed in 2019. Find extensive documentation
[here](https://godoc.org/github.com/hanwen/go-fuse/).
[here](https://godoc.org/github.com/hanwen/go-fuse/v2).
Further highlights of this library is
......
all.bash
View file @ 43d5f40d
#!/bin/sh
set -eu
#!/bin/bash
set -eux
for d in fuse fuse/nodefs fuse/pathfs fuse/test zipfs unionfs \
example/hello example/loopback example/zipfs \
example/multizip example/unionfs example/memfs \
example/autounionfs example/statfs ; \
do
go build -o /dev/null github.com/hanwen/go-fuse/${d}
done
# Everything must compile on Linux
go build ./...
# Not everything compiles on MacOS (try GOOS=darwin go build ./...).
# But our key packages should.
GOOS=darwin go build ./fuse/... ./fs/... ./example/loopback/...
for d in fuse zipfs unionfs fuse/test
do
(
cd $d
# Make sure it compiles on all platforms.
for GOOS in darwin linux ; do
export GOOS
go test -c -i github.com/hanwen/go-fuse/$d
done
echo "go test github.com/hanwen/go-fuse/$d"
go test github.com/hanwen/go-fuse/$d
echo "go test -race github.com/hanwen/go-fuse/$d"
go test -race github.com/hanwen/go-fuse/$d
)
done
for target in "clean" "install" ; do
for d in fuse fuse/nodefs fuse/pathfs fuse/test zipfs unionfs \
example/hello example/loopback example/zipfs \
example/multizip example/unionfs example/memfs \
example/autounionfs example/statfs ; \
do
if test "${target}" = "install" && test "${d}" = "fuse/test"; then
continue
fi
echo "go ${target} github.com/hanwen/go-fuse/${d}"
go ${target} github.com/hanwen/go-fuse/${d}
done
done
# Run the tests. Why the flags:
# -timeout 5m ... Get a backtrace on a hung test before the CI system kills us
# -p 1 .......... Run tests serially, which also means we get live output
# instead of per-package buffering.
# -count 1 ...... Disable result caching, so we can see flakey tests
go test -timeout 5m -p 1 -count 1 ./...
make -C benchmark
for d in benchmark
do
go test github.com/hanwen/go-fuse/benchmark -test.bench '.*' -test.cpu 1,2
done
go test ./benchmark -test.bench '.*' -test.cpu 1,2
benchmark/read_test.go 0 → 100644
View file @ 43d5f40d
// Copyright 2021 the Go-FUSE Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package benchmark
import (
"fmt"
"os"
"os/exec"
"testing"
"github.com/hanwen/go-fuse/v2/internal/testutil"
"golang.org/x/sync/errgroup"
)
func BenchmarkGoFuseRead(b *testing.B) {
fs := &readFS{}
wd, clean := setupFs(fs, b.N)
defer clean()
jobs := 32
blockSize := 64 * 1024
cmds := make([]*exec.Cmd, jobs)
for i := 0; i < jobs; i++ {
cmds[i] = exec.Command("dd",
fmt.Sprintf("if=%s/foo.txt", wd),
"iflag=direct",
"of=/dev/null",
fmt.Sprintf("bs=%d", blockSize),
fmt.Sprintf("count=%d", b.N))
if testutil.VerboseTest() {
cmds[i].Stdout = os.Stdout
cmds[i].Stderr = os.Stderr
}
}
b.SetBytes(int64(jobs * blockSize))
b.ReportAllocs()
b.ResetTimer()
var eg errgroup.Group
for i := 0; i < jobs; i++ {
i := i
eg.Go(func() error {
return cmds[i].Run()
})
}
if err := eg.Wait(); err != nil {
b.Fatalf("dd failed: %v", err)
}
b.StopTimer()
}
benchmark/readfs.go 0 → 100644
View file @ 43d5f40d
// Copyright 2021 the Go-FUSE Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package benchmark
import (
"context"
"syscall"
"time"
"github.com/hanwen/go-fuse/v2/fs"
"github.com/hanwen/go-fuse/v2/fuse"
)
const fileSize = 2 << 60
// readFS is a filesystem that always and immediately returns zeros on read
// operations. Useful when benchmarking the raw throughput with go-fuse.
type readFS struct {
fs.Inode
}
var _ = (fs.NodeLookuper)((*readFS)(nil))
func (n *readFS) Lookup(ctx context.Context, name string, out *fuse.EntryOut) (*fs.Inode, syscall.Errno) {
sattr := fs.StableAttr{Mode: fuse.S_IFREG}
return n.NewInode(ctx, &readFS{}, sattr), fs.OK
}
var _ = (fs.NodeGetattrer)((*readFS)(nil))
func (n *readFS) Getattr(ctx context.Context, f fs.FileHandle, out *fuse.AttrOut) syscall.Errno {
out.Size = fileSize
out.SetTimeout(time.Hour)
return fs.OK
}
var _ = (fs.NodeOpener)((*readFS)(nil))
func (n *readFS) Open(ctx context.Context, flags uint32) (fh fs.FileHandle, fuseFlags uint32, errno syscall.Errno) {
return &readFS{}, fuse.FOPEN_DIRECT_IO, fs.OK
}
var _ = (fs.FileReader)((*readFS)(nil))
func (n *readFS) Read(ctx context.Context, dest []byte, offset int64) (fuse.ReadResult, syscall.Errno) {
return fuse.ReadResultData(dest), fs.OK
}
benchmark/stat_test.go
View file @ 43d5f40d
......@@ -129,7 +129,7 @@ func BenchmarkGoFuseStat(b *testing.B) {
threads := runtime.GOMAXPROCS(0)
if err := TestingBOnePass(b, threads, fileList, wd); err != nil {
log.Fatalf("TestingBOnePass %v8", err)
b.Fatalf("TestingBOnePass %v8", err)
}
}
......@@ -252,6 +252,8 @@ func BenchmarkCFuseThreadedStat(b *testing.B) {
f.Close()
mountPoint := testutil.TempDir()
defer os.RemoveAll(mountPoint)
cmd := exec.Command(wd+"/cstatfs",
"-o",
"entry_timeout=0.0,attr_timeout=0.0,ac_attr_timeout=0.0,negative_timeout=0.0",
......@@ -274,6 +276,6 @@ func BenchmarkCFuseThreadedStat(b *testing.B) {
os.Lstat(mountPoint)
threads := runtime.GOMAXPROCS(0)
if err := TestingBOnePass(b, threads, fileList, mountPoint); err != nil {
log.Fatalf("TestingBOnePass %v", err)
b.Fatalf("TestingBOnePass %v", err)
}
}
example/autounionfs/main.go deleted 100644 → 0
View file @ 51ae026e
// Copyright 2016 the Go-FUSE Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package main
import (
"flag"
"fmt"
"os"
"time"
"github.com/hanwen/go-fuse/v2/fuse"
"github.com/hanwen/go-fuse/v2/fuse/nodefs"
"github.com/hanwen/go-fuse/v2/fuse/pathfs"
"github.com/hanwen/go-fuse/v2/unionfs"
)
func main() {
debug := flag.Bool("debug", false, "debug on")
hardlinks := flag.Bool("hardlinks", false, "support hardlinks")
delcache_ttl := flag.Float64("deletion_cache_ttl", 5.0, "Deletion cache TTL in seconds.")
branchcache_ttl := flag.Float64("branchcache_ttl", 5.0, "Branch cache TTL in seconds.")
deldirname := flag.String(
"deletion_dirname", "GOUNIONFS_DELETIONS", "Directory name to use for deletions.")
hide_readonly_link := flag.Bool("hide_readonly_link", true,
"Hides READONLY link from the top mountpoints. "+
"Enabled by default.")
portableInodes := flag.Bool("portable-inodes", false,
"Use sequential 32-bit inode numbers.")
flag.Parse()
if len(flag.Args()) < 2 {
fmt.Println("Usage:\n main MOUNTPOINT BASEDIR")
os.Exit(2)
}
ufsOptions := unionfs.UnionFsOptions{
DeletionCacheTTL: time.Duration(*delcache_ttl * float64(time.Second)),
BranchCacheTTL: time.Duration(*branchcache_ttl * float64(time.Second)),
DeletionDirName: *deldirname,
}
options := unionfs.AutoUnionFsOptions{
UnionFsOptions: ufsOptions,
Options: nodefs.Options{
EntryTimeout: time.Second,
AttrTimeout: time.Second,
NegativeTimeout: time.Second,
Owner: fuse.CurrentOwner(),
Debug: *debug,
},
UpdateOnMount: true,
PathNodeFsOptions: pathfs.PathNodeFsOptions{
ClientInodes: *hardlinks,
},
HideReadonly: *hide_readonly_link,
}
fsOpts := nodefs.Options{
PortableInodes: *portableInodes,
Debug: *debug,
}
gofs := unionfs.NewAutoUnionFs(flag.Arg(1), options)
pathfs := pathfs.NewPathNodeFs(gofs, &pathfs.PathNodeFsOptions{
Debug: *debug,
})
state, _, err := nodefs.MountRoot(flag.Arg(0), pathfs.Root(), &fsOpts)
if err != nil {
fmt.Printf("Mount fail: %v\n", err)
os.Exit(1)
}
state.Serve()
time.Sleep(1 * time.Second)
}
example/unionfs/main.go deleted 100644 → 0
View file @ 51ae026e
// Copyright 2016 the Go-FUSE Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package main
import (
"flag"
"fmt"
"log"
"os"
"time"
"github.com/hanwen/go-fuse/v2/fuse/nodefs"
"github.com/hanwen/go-fuse/v2/fuse/pathfs"
"github.com/hanwen/go-fuse/v2/unionfs"
)
func main() {
debug := flag.Bool("debug", false, "debug on")
portable := flag.Bool("portable", false, "use 32 bit inodes")
entry_ttl := flag.Float64("entry_ttl", 1.0, "fuse entry cache TTL.")
negative_ttl := flag.Float64("negative_ttl", 1.0, "fuse negative entry cache TTL.")
delcache_ttl := flag.Float64("deletion_cache_ttl", 5.0, "Deletion cache TTL in seconds.")
branchcache_ttl := flag.Float64("branchcache_ttl", 5.0, "Branch cache TTL in seconds.")
deldirname := flag.String(
"deletion_dirname", "GOUNIONFS_DELETIONS", "Directory name to use for deletions.")
flag.Parse()
if len(flag.Args()) < 2 {
fmt.Println("Usage:\n unionfs MOUNTPOINT RW-DIRECTORY RO-DIRECTORY ...")
os.Exit(2)
}
ufsOptions := unionfs.UnionFsOptions{
DeletionCacheTTL: time.Duration(*delcache_ttl * float64(time.Second)),
BranchCacheTTL: time.Duration(*branchcache_ttl * float64(time.Second)),
DeletionDirName: *deldirname,
}
ufs, err := unionfs.NewUnionFsFromRoots(flag.Args()[1:], &ufsOptions, true)
if err != nil {
log.Fatal("Cannot create UnionFs", err)
os.Exit(1)
}
nodeFs := pathfs.NewPathNodeFs(ufs, &pathfs.PathNodeFsOptions{ClientInodes: true})
mOpts := nodefs.Options{
EntryTimeout: time.Duration(*entry_ttl * float64(time.Second)),
AttrTimeout: time.Duration(*entry_ttl * float64(time.Second)),
NegativeTimeout: time.Duration(*negative_ttl * float64(time.Second)),
PortableInodes: *portable,
Debug: *debug,
}
mountState, _, err := nodefs.MountRoot(flag.Arg(0), nodeFs.Root(), &mOpts)
if err != nil {
log.Fatal("Mount fail:", err)
}
mountState.Serve()
}
example/winfs/main.go 0 → 100644
View file @ 43d5f40d
// Copyright 2020 the Go-FUSE Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// This is main program driver for a loopback filesystem that emulates
// windows semantics (no delete/rename on opened files.)
package main
import (
"context"
"flag"
"fmt"
"log"
"os"
"path"
"sync"
"syscall"
"time"
"github.com/hanwen/go-fuse/v2/fs"
"github.com/hanwen/go-fuse/v2/fuse"
)
// Release decreases the open count. The kernel doesn't wait with
// returning from close(), so if the caller is too quick to
// unlink/rename after calling close(), this may still trigger EBUSY.
// Kludge around this by sleeping for a bit before we check business.
var delay = flag.Duration("delay", 10*time.Microsecond,
"wait this long before checking business")
// WindowsNode is a loopback FS node keeping track of open counts.
type WindowsNode struct {
// WindowsNode inherits most functionality from LoopbackNode.
fs.LoopbackNode
mu sync.Mutex
openCount int
}
func (n *WindowsNode) increment() {
n.mu.Lock()
defer n.mu.Unlock()
n.openCount++
}
func (n *WindowsNode) decrement() {
n.mu.Lock()
defer n.mu.Unlock()
n.openCount--
}
var _ = (fs.NodeOpener)((*WindowsNode)(nil))
func (n *WindowsNode) Open(ctx context.Context, flags uint32) (fs.FileHandle, uint32, syscall.Errno) {
fh, flags, errno := n.LoopbackNode.Open(ctx, flags)
if errno == 0 {
n.increment()
}
return fh, flags, errno
}
var _ = (fs.NodeCreater)((*WindowsNode)(nil))
func (n *WindowsNode) Create(ctx context.Context, name string, flags uint32, mode uint32, out *fuse.EntryOut) (*fs.Inode, fs.FileHandle, uint32, syscall.Errno) {
inode, fh, flags, errno := n.LoopbackNode.Create(ctx, name, flags, mode, out)
if errno == 0 {
wn := inode.Operations().(*WindowsNode)
wn.increment()
}
return inode, fh, flags, errno
}
var _ = (fs.NodeReleaser)((*WindowsNode)(nil))
func (n *WindowsNode) Release(ctx context.Context, f fs.FileHandle) syscall.Errno {
n.decrement()
if fr, ok := f.(fs.FileReleaser); ok {
return fr.Release(ctx)
}
return 0
}
func isBusy(parent *fs.Inode, name string) bool {
time.Sleep(*delay)
if ch := parent.GetChild(name); ch != nil {
if wn, ok := ch.Operations().(*WindowsNode); ok {
wn.mu.Lock()
defer wn.mu.Unlock()
if wn.openCount > 0 {
return true
}
}
}
return false
}
var _ = (fs.NodeUnlinker)((*WindowsNode)(nil))
func (n *WindowsNode) Unlink(ctx context.Context, name string) syscall.Errno {
if isBusy(n.EmbeddedInode(), name) {
return syscall.EBUSY
}
return n.LoopbackNode.Unlink(ctx, name)
}
var _ = (fs.NodeRenamer)((*WindowsNode)(nil))
func (n *WindowsNode) Rename(ctx context.Context, name string, newParent fs.InodeEmbedder, newName string, flags uint32) syscall.Errno {
if isBusy(n.EmbeddedInode(), name) || isBusy(newParent.EmbeddedInode(), newName) {
return syscall.EBUSY
}
return n.LoopbackNode.Rename(ctx, name, newParent, newName, flags)
}
func newWindowsNode(rootData *fs.LoopbackRoot, _ *fs.Inode, _ string, _ *syscall.Stat_t) fs.InodeEmbedder {
n := &WindowsNode{
LoopbackNode: fs.LoopbackNode{
RootData: rootData,
},
}
return n
}
func main() {
log.SetFlags(log.Lmicroseconds)
debug := flag.Bool("debug", false, "print debugging messages.")
flag.Parse()
if flag.NArg() < 2 {
fmt.Printf("usage: %s MOUNTPOINT ORIGINAL\n", path.Base(os.Args[0]))
fmt.Printf("\noptions:\n")
flag.PrintDefaults()
os.Exit(2)
}
orig := flag.Arg(1)
rootData := &fs.LoopbackRoot{
NewNode: newWindowsNode,
Path: orig,
}
sec := time.Second
opts := &fs.Options{
AttrTimeout: &sec,
EntryTimeout: &sec,
}
opts.Debug = *debug
opts.MountOptions.Options = append(opts.MountOptions.Options, "fsname="+orig)
opts.MountOptions.Name = "winfs"
opts.NullPermissions = true
server, err := fs.Mount(flag.Arg(0), newWindowsNode(rootData, nil, "", nil), opts)
if err != nil {
log.Fatalf("Mount fail: %v\n", err)
}
fmt.Println("Mounted!")
server.Wait()
}
fs/bridge.go
View file @ 43d5f40d
......@@ -7,7 +7,7 @@ package fs
import (
"context"
"log"
"math/rand"
"runtime/debug"
"sync"
"syscall"
"time"
......@@ -61,10 +61,34 @@ type rawBridge struct {
// mu protects the following data. Locks for inodes must be
// taken before rawBridge.mu
mu sync.Mutex
nodes map[uint64]*Inode
mu sync.Mutex
// stableAttrs is used to detect already-known nodes and hard links by
// looking at:
// 1) file type ......... StableAttr.Mode
// 2) inode number ...... StableAttr.Ino
// 3) generation number . StableAttr.Gen
stableAttrs map[StableAttr]*Inode
automaticIno uint64
// The *Node ID* is an arbitrary uint64 identifier chosen by the FUSE library.
// It is used the identify *nodes* (files/directories/symlinks/...) in the
// communication between the FUSE library and the Linux kernel.
//
// The kernelNodeIds map translates between the NodeID and the corresponding
// go-fuse Inode object.
//
// A simple incrementing counter is used as the NodeID (see `nextNodeID`).
kernelNodeIds map[uint64]*Inode
// nextNodeID is the next free NodeID. Increment after copying the value.
nextNodeId uint64
// nodeCountHigh records the highest number of entries we had in the
// kernelNodeIds map.
// As the size of stableAttrs tracks kernelNodeIds (+- a few entries due to
// concurrent FORGETs, LOOKUPs, and the fixed NodeID 1), this is also a good
// estimate for stableAttrs.
nodeCountHigh int
files []*fileEntry
freeFiles []uint32
}
......@@ -83,55 +107,26 @@ func (b *rawBridge) newInodeUnlocked(ops InodeEmbedder, id StableAttr, persisten
return ops.embed()
}
// Only the file type bits matter
id.Mode = id.Mode & syscall.S_IFMT
if id.Mode == 0 {
id.Mode = fuse.S_IFREG
}
if id.Ino == 0 {
// Find free inode number.
for {
id.Ino = b.automaticIno
b.automaticIno++
_, ok := b.nodes[id.Ino]
_, ok := b.stableAttrs[id]
if !ok {
break
}
}
}
// Only the file type bits matter
id.Mode = id.Mode & syscall.S_IFMT
if id.Mode == 0 {
id.Mode = fuse.S_IFREG
}
// the same node can be looked up through 2 paths in parallel, eg.
//
// root
// / \
// dir1 dir2
// \ /
// file
//
// dir1.Lookup("file") and dir2.Lookup("file") are executed
// simultaneously. The matching StableAttrs ensure that we return the
// same node.
var t time.Duration
t0 := time.Now()
for i := 1; true; i++ {
old := b.nodes[id.Ino]
if old == nil {
break
}
if old.stableAttr == id {
return old
}
b.mu.Unlock()
t = expSleep(t)
if i%5000 == 0 {
b.logf("blocked for %.0f seconds waiting for FORGET on i%d", time.Since(t0).Seconds(), id.Ino)
}
b.mu.Lock()
}
b.nodes[id.Ino] = ops.embed()
initInode(ops.embed(), ops, id, b, persistent)
initInode(ops.embed(), ops, id, b, persistent, b.nextNodeId)
b.nextNodeId++
return ops.embed()
}
......@@ -141,21 +136,6 @@ func (b *rawBridge) logf(format string, args ...interface{}) {
}
}
// expSleep sleeps for time `t` and returns an exponentially increasing value
// for the next sleep time, capped at 1 ms.
func expSleep(t time.Duration) time.Duration {
if t == 0 {
return time.Microsecond
}
time.Sleep(t)
// Next sleep is between t and 2*t
t += time.Duration(rand.Int63n(int64(t)))
if t >= time.Millisecond {
return time.Millisecond
}
return t
}
func (b *rawBridge) newInode(ctx context.Context, ops InodeEmbedder, id StableAttr, persistent bool) *Inode {
ch := b.newInodeUnlocked(ops, id, persistent)
if ch != ops.embed() {
......@@ -169,34 +149,84 @@ func (b *rawBridge) newInode(ctx context.Context, ops InodeEmbedder, id StableAt
}
// addNewChild inserts the child into the tree. Returns file handle if file != nil.
func (b *rawBridge) addNewChild(parent *Inode, name string, child *Inode, file FileHandle, fileFlags uint32, out *fuse.EntryOut) uint32 {
// Unless fileFlags has the syscall.O_EXCL bit set, child.stableAttr will be used
// to find an already-known node. If one is found, `child` is ignored and the
// already-known one is used. The node that was actually used is returned.
func (b *rawBridge) addNewChild(parent *Inode, name string, child *Inode, file FileHandle, fileFlags uint32, out *fuse.EntryOut) (selected *Inode, fh uint32) {
if name == "." || name == ".." {
log.Panicf("BUG: tried to add virtual entry %q to the actual tree", name)
}
lockNodes(parent, child)
parent.setEntry(name, child)
b.mu.Lock()
// Due to concurrent FORGETs, lookupCount may have dropped to zero.
// This means it MAY have been deleted from nodes[] already. Add it back.
if child.lookupCount == 0 {
b.nodes[child.stableAttr.Ino] = child
// the same node can be looked up through 2 paths in parallel, eg.
//
// root
// / \
// dir1 dir2
// \ /
// file
//
// dir1.Lookup("file") and dir2.Lookup("file") are executed
// simultaneously. The matching StableAttrs ensure that we return the
// same node.
orig := child
id := child.stableAttr
if id.Mode & ^(uint32(syscall.S_IFMT)) != 0 {
log.Panicf("%#v", id)
}
for {
lockNodes(parent, child)
b.mu.Lock()
if fileFlags&syscall.O_EXCL != 0 {
// must create a new node - don't look for existing nodes
break
}
old := b.stableAttrs[id]
if old == nil {
if child == orig {
// no pre-existing node under this inode number
break
} else {
// old inode disappeared while we were looping here. Go back to
// original child.
b.mu.Unlock()
unlockNodes(parent, child)
child = orig
continue
}
}
if old == child {
// we now have the right inode locked
break
}
// found a different existing node
b.mu.Unlock()
unlockNodes(parent, child)
child = old
}
child.lookupCount++
child.changeCounter++
var fh uint32
b.kernelNodeIds[child.nodeId] = child
if len(b.kernelNodeIds) > b.nodeCountHigh {
b.nodeCountHigh = len(b.kernelNodeIds)
}
// Any node that might be there is overwritten - it is obsolete now
b.stableAttrs[id] = child
if file != nil {
fh = b.registerFile(child, file, fileFlags)
}
out.NodeId = child.stableAttr.Ino
parent.setEntry(name, child)
out.NodeId = child.nodeId
out.Generation = child.stableAttr.Gen
out.Attr.Ino = child.stableAttr.Ino
b.mu.Unlock()
unlockNodes(parent, child)
return fh
return child, fh
}
func (b *rawBridge) setEntryOutTimeout(out *fuse.EntryOut) {
......@@ -237,9 +267,8 @@ func NewNodeFS(root InodeEmbedder, opts *Options) fuse.RawFileSystem {
bridge := &rawBridge{
automaticIno: opts.FirstAutomaticIno,
server: opts.ServerCallbacks,
}
if bridge.automaticIno == 1 {
bridge.automaticIno++
nextNodeId: 2, // the root node has nodeid 1
stableAttrs: make(map[StableAttr]*Inode),
}
if bridge.automaticIno == 0 {
......@@ -256,15 +285,16 @@ func NewNodeFS(root InodeEmbedder, opts *Options) fuse.RawFileSystem {
initInode(root.embed(), root,
StableAttr{
Ino: 1,
Ino: root.embed().StableAttr().Ino,
Mode: fuse.S_IFDIR,
},
bridge,
false,
1,
)
bridge.root = root.embed()
bridge.root.lookupCount = 1
bridge.nodes = map[uint64]*Inode{
bridge.kernelNodeIds = map[uint64]*Inode{
1: bridge.root,
}
......@@ -287,7 +317,7 @@ func (b *rawBridge) String() string {
func (b *rawBridge) inode(id uint64, fh uint64) (*Inode, *fileEntry) {
b.mu.Lock()
defer b.mu.Unlock()
n, f := b.nodes[id], b.files[fh]
n, f := b.kernelNodeIds[id], b.files[fh]
if n == nil {
log.Panicf("unknown node %d", id)
}
......@@ -306,8 +336,8 @@ func (b *rawBridge) Lookup(cancel <-chan struct{}, header *fuse.InHeader, name s
return errnoToStatus(errno)
}
child, _ = b.addNewChild(parent, name, child, nil, 0, out)
child.setEntryOut(out)
b.addNewChild(parent, name, child, nil, 0, out)
b.setEntryOutTimeout(out)
return fuse.OK
}
......@@ -382,8 +412,8 @@ func (b *rawBridge) Mkdir(cancel <-chan struct{}, input *fuse.MkdirIn, name stri
log.Panicf("Mkdir: mode must be S_IFDIR (%o), got %o", fuse.S_IFDIR, out.Attr.Mode)
}
child, _ = b.addNewChild(parent, name, child, nil, syscall.O_EXCL, out)
child.setEntryOut(out)
b.addNewChild(parent, name, child, nil, 0, out)
b.setEntryOutTimeout(out)
return fuse.OK
}
......@@ -395,14 +425,16 @@ func (b *rawBridge) Mknod(cancel <-chan struct{}, input *fuse.MknodIn, name stri
var errno syscall.Errno
if mops, ok := parent.ops.(NodeMknoder); ok {
child, errno = mops.Mknod(&fuse.Context{Caller: input.Caller, Cancel: cancel}, name, input.Mode, input.Rdev, out)
} else {
return fuse.ENOTSUP
}
if errno != 0 {
return errnoToStatus(errno)
}
child, _ = b.addNewChild(parent, name, child, nil, syscall.O_EXCL, out)
child.setEntryOut(out)
b.addNewChild(parent, name, child, nil, 0, out)
b.setEntryOutTimeout(out)
return fuse.OK
}
......@@ -428,8 +460,9 @@ func (b *rawBridge) Create(cancel <-chan struct{}, input *fuse.CreateIn, name st
return errnoToStatus(errno)
}
out.Fh = uint64(b.addNewChild(parent, name, child, f, input.Flags|syscall.O_CREAT, &out.EntryOut))
child, fh := b.addNewChild(parent, name, child, f, input.Flags|syscall.O_CREAT|syscall.O_EXCL, &out.EntryOut)
out.Fh = uint64(fh)
out.OpenFlags = flags
child.setEntryOut(&out.EntryOut)
......@@ -439,7 +472,47 @@ func (b *rawBridge) Create(cancel <-chan struct{}, input *fuse.CreateIn, name st
func (b *rawBridge) Forget(nodeid, nlookup uint64) {
n, _ := b.inode(nodeid, 0)
n.removeRef(nlookup, false)
forgotten, _ := n.removeRef(nlookup, false)
if forgotten {
b.compactMemory()
}
}
// compactMemory tries to free memory that was previously used by forgotten
// nodes.
//
// Maps do not free all memory when elements get deleted
// ( https://github.com/golang/go/issues/20135 ).
// As a workaround, we recreate our two big maps (stableAttrs & kernelNodeIds)
// every time they have shrunk dramatically (100 x smaller).
// In this case, `nodeCountHigh` is reset to the new (smaller) size.
func (b *rawBridge) compactMemory() {
b.mu.Lock()
if b.nodeCountHigh <= len(b.kernelNodeIds)*100 {
b.mu.Unlock()
return
}
tmpStableAttrs := make(map[StableAttr]*Inode, len(b.stableAttrs))
for i, v := range b.stableAttrs {
tmpStableAttrs[i] = v
}
b.stableAttrs = tmpStableAttrs
tmpKernelNodeIds := make(map[uint64]*Inode, len(b.kernelNodeIds))
for i, v := range b.kernelNodeIds {
tmpKernelNodeIds[i] = v
}
b.kernelNodeIds = tmpKernelNodeIds
b.nodeCountHigh = len(b.kernelNodeIds)
b.mu.Unlock()
// Run outside b.mu
debug.FreeOSMemory()
}
func (b *rawBridge) SetDebug(debug bool) {}
......@@ -540,8 +613,8 @@ func (b *rawBridge) Link(cancel <-chan struct{}, input *fuse.LinkIn, name string
return errnoToStatus(errno)
}
child, _ = b.addNewChild(parent, name, child, nil, 0, out)
child.setEntryOut(out)
b.addNewChild(parent, name, child, nil, 0, out)
b.setEntryOutTimeout(out)
return fuse.OK
}
......@@ -557,7 +630,7 @@ func (b *rawBridge) Symlink(cancel <-chan struct{}, header *fuse.InHeader, targe
return errnoToStatus(status)
}
b.addNewChild(parent, name, child, nil, 0, out)
child, _ = b.addNewChild(parent, name, child, nil, syscall.O_EXCL, out)
child.setEntryOut(out)
b.setEntryOutTimeout(out)
return fuse.OK
......@@ -769,7 +842,7 @@ func (b *rawBridge) releaseFileEntry(nid uint64, fh uint64) (*Inode, *fileEntry)
b.mu.Lock()
defer b.mu.Unlock()
n := b.nodes[nid]
n := b.kernelNodeIds[nid]
var entry *fileEntry
if fh > 0 {
last := len(n.openFiles) - 1
......@@ -994,7 +1067,7 @@ func (b *rawBridge) ReadDirPlus(cancel <-chan struct{}, input *fuse.ReadIn, out
entryOut.SetEntryTimeout(*b.options.NegativeTimeout)
}
} else {
b.addNewChild(n, e.Name, child, nil, 0, entryOut)
child, _ = b.addNewChild(n, e.Name, child, nil, 0, entryOut)
child.setEntryOut(entryOut)
b.setEntryOutTimeout(entryOut)
if e.Mode&syscall.S_IFMT != child.stableAttr.Mode&syscall.S_IFMT {
......
fs/bridge_test.go
View file @ 43d5f40d
......@@ -201,3 +201,40 @@ func (n *testDeletedIno) Getattr(ctx context.Context, f FileHandle, out *fuse.At
// Otherwise EILSEQ
return syscall.EILSEQ
}
// TestIno1 tests that inode number 1 is allowed.
//
// We used to panic like this because inode number 1 was special:
//
// panic: using reserved ID 1 for inode number
//
func TestIno1(t *testing.T) {
rootNode := testIno1{}
mnt, _, clean := testMount(t, &rootNode, nil)
defer clean()
var st syscall.Stat_t
err := syscall.Stat(mnt+"/ino1", &st)
if err != nil {
t.Fatal(err)
}
if st.Ino != 1 {
t.Errorf("wrong inode number: want=1 have=%d", st.Ino)
}
}
type testIno1 struct {
Inode
}
func (fn *testIno1) Lookup(ctx context.Context, name string, out *fuse.EntryOut) (*Inode, syscall.Errno) {
if name != "ino1" {
return nil, syscall.ENOENT
}
stable := StableAttr{
Mode: syscall.S_IFREG,
Ino: 1,
}
child := fn.NewInode(ctx, &testIno1{}, stable)
return child, 0
}
fs/dirstream_linux.go
View file @ 43d5f40d
......@@ -55,12 +55,26 @@ func (ds *loopbackDirStream) HasNext() bool {
return len(ds.todo) > 0
}
// Like syscall.Dirent, but without the [256]byte name.
type dirent struct {
Ino uint64
Off int64
Reclen uint16
Type uint8
Name [1]uint8 // align to 4 bytes for 32 bits.
}
func (ds *loopbackDirStream) Next() (fuse.DirEntry, syscall.Errno) {
ds.mu.Lock()
defer ds.mu.Unlock()
de := (*syscall.Dirent)(unsafe.Pointer(&ds.todo[0]))
nameBytes := ds.todo[unsafe.Offsetof(syscall.Dirent{}.Name):de.Reclen]
// We can't use syscall.Dirent here, because it declares a
// [256]byte name, which may run beyond the end of ds.todo.
// when that happens in the race detector, it causes a panic
// "converted pointer straddles multiple allocations"
de := (*dirent)(unsafe.Pointer(&ds.todo[0]))
nameBytes := ds.todo[unsafe.Offsetof(dirent{}.Name):de.Reclen]
ds.todo = ds.todo[de.Reclen:]
// After the loop, l contains the index of the first '\0'.
......
fs/files.go
View file @ 43d5f40d
......@@ -17,7 +17,9 @@ import (
)
// NewLoopbackFile creates a FileHandle out of a file descriptor. All
// operations are implemented.
// operations are implemented. When using the Fd from a *os.File, call
// syscall.Dup() on the fd, to avoid os.File's finalizer from closing
// the file descriptor.
func NewLoopbackFile(fd int) FileHandle {
return &loopbackFile{fd: fd}
}
......
fs/forget_test.go
View file @ 43d5f40d
......@@ -112,7 +112,7 @@ func TestForget(t *testing.T) {
bridge := rawFS.(*rawBridge)
bridge.mu.Lock()
l := len(bridge.nodes)
l := len(bridge.kernelNodeIds)
bridge.mu.Unlock()
if l != 1 {
t.Fatalf("got %d live nodes, want 1", l)
......
fs/inode.go
View file @ 43d5f40d
......@@ -18,11 +18,6 @@ import (
"github.com/hanwen/go-fuse/v2/fuse"
)
type parentData struct {
name string
parent *Inode
}
// StableAttr holds immutable attributes of a object in the filesystem.
type StableAttr struct {
// Each Inode has a type, which does not change over the
......@@ -32,8 +27,8 @@ type StableAttr struct {
// The inode number must be unique among the currently live
// objects in the file system. It is used to communicate to
// the kernel about this file object. The values uint64(-1),
// and 1 are reserved. When using Ino==0, a unique, sequential
// the kernel about this file object. The value uint64(-1)
// is reserved. When using Ino==0, a unique, sequential
// number is assigned (starting at 2^63 by default) on Inode creation.
Ino uint64
......@@ -46,7 +41,7 @@ type StableAttr struct {
// Reserved returns if the StableAttr is using reserved Inode numbers.
func (i *StableAttr) Reserved() bool {
return i.Ino == 1 || i.Ino == ^uint64(0)
return i.Ino == ^uint64(0) // fuse.pollHackInode = ^uint64(0)
}
// Inode is a node in VFS tree. Inodes are one-to-one mapped to
......@@ -63,6 +58,11 @@ type Inode struct {
ops InodeEmbedder
bridge *rawBridge
// The *Node ID* is an arbitrary uint64 identifier chosen by the FUSE library.
// It is used the identify *nodes* (files/directories/symlinks/...) in the
// communication between the FUSE library and the Linux kernel.
nodeId uint64
// Following data is mutable.
// file handles.
......@@ -100,11 +100,11 @@ type Inode struct {
// Parents of this Inode. Can be more than one due to hard links.
// When you change this, you MUST increment changeCounter.
parents map[parentData]struct{}
parents inodeParents
}
func (n *Inode) IsDir() bool {
return n.stableAttr.Mode&syscall.S_IFDIR != 0
return n.stableAttr.Mode&syscall.S_IFMT == syscall.S_IFDIR
}
func (n *Inode) embed() *Inode {
......@@ -115,12 +115,12 @@ func (n *Inode) EmbeddedInode() *Inode {
return n
}
func initInode(n *Inode, ops InodeEmbedder, attr StableAttr, bridge *rawBridge, persistent bool) {
func initInode(n *Inode, ops InodeEmbedder, attr StableAttr, bridge *rawBridge, persistent bool, nodeId uint64) {
n.ops = ops
n.stableAttr = attr
n.bridge = bridge
n.persistent = persistent
n.parents = make(map[parentData]struct{})
n.nodeId = nodeId
if attr.Mode == fuse.S_IFDIR {
n.children = make(map[string]*Inode)
}
......@@ -128,7 +128,7 @@ func initInode(n *Inode, ops InodeEmbedder, attr StableAttr, bridge *rawBridge,
// Set node ID and mode in EntryOut
func (n *Inode) setEntryOut(out *fuse.EntryOut) {
out.NodeId = n.stableAttr.Ino
out.NodeId = n.nodeId
out.Ino = n.stableAttr.Ino
out.Mode = (out.Attr.Mode & 07777) | n.stableAttr.Mode
}
......@@ -259,7 +259,7 @@ func unlockNodes(ns ...*Inode) {
func (n *Inode) Forgotten() bool {
n.mu.Lock()
defer n.mu.Unlock()
return n.lookupCount == 0 && len(n.parents) == 0 && !n.persistent
return n.lookupCount == 0 && n.parents.count() == 0 && !n.persistent
}
// Operations returns the object implementing the file system
......@@ -277,34 +277,24 @@ func (n *Inode) Path(root *Inode) string {
var segments []string
p := n
for p != nil && p != root {
var pd parentData
// We don't try to take all locks at the same time, because
// the caller won't use the "path" string under lock anyway.
found := false
p.mu.Lock()
// Select an arbitrary parent
for pd = range p.parents {
found = true
break
}
// Get last known parent
pd := p.parents.get()
p.mu.Unlock()
if found == false {
if pd == nil {
p = nil
break
}
if pd.parent == nil {
break
}
segments = append(segments, pd.name)
p = pd.parent
}
if root != nil && root != p {
deletedPlaceholder := fmt.Sprintf(".go-fuse.%d/deleted", rand.Uint64())
n.bridge.logf("warning: Inode.Path: inode i%d is orphaned, replacing segment with %q",
n.stableAttr.Ino, deletedPlaceholder)
n.bridge.logf("warning: Inode.Path: n%d is orphaned, replacing segment with %q",
n.nodeId, deletedPlaceholder)
// NOSUBMIT - should replace rather than append?
segments = append(segments, deletedPlaceholder)
}
......@@ -329,7 +319,14 @@ func (n *Inode) Path(root *Inode) string {
// but it could be also valid if only iparent is locked and ichild was just
// created and only one goroutine keeps referencing it.
func (iparent *Inode) setEntry(name string, ichild *Inode) {
ichild.parents[parentData{name, iparent}] = struct{}{}
newParent := parentData{name, iparent}
if ichild.stableAttr.Mode == syscall.S_IFDIR {
// Directories cannot have more than one parent. Clear the map.
// This special-case is neccessary because ichild may still have a
// parent that was forgotten (i.e. removed from bridge.inoMap).
ichild.parents.clear()
}
ichild.parents.add(newParent)
iparent.children[name] = ichild
ichild.changeCounter++
iparent.changeCounter++
......@@ -360,8 +357,8 @@ func (n *Inode) ForgetPersistent() {
// NewInode returns an inode for the given InodeEmbedder. The mode
// should be standard mode argument (eg. S_IFDIR). The inode number in
// id.Ino argument is used to implement hard-links. If it is given,
// and another node with the same ID is known, that will node will be
// returned, and the passed-in `node` is ignored.
// and another node with the same ID is known, the new inode may be
// ignored, and the old one used instead.
func (n *Inode) NewInode(ctx context.Context, node InodeEmbedder, id StableAttr) *Inode {
return n.newInode(ctx, node, id, false)
}
......@@ -380,8 +377,9 @@ func (n *Inode) removeRef(nlookup uint64, dropPersistence bool) (forgotten bool,
n.mu.Lock()
if nlookup > 0 && dropPersistence {
log.Panic("only one allowed")
} else if nlookup > n.lookupCount {
log.Panicf("n%d lookupCount underflow: lookupCount=%d, decrement=%d", n.nodeId, n.lookupCount, nlookup)
} else if nlookup > 0 {
n.lookupCount -= nlookup
n.changeCounter++
} else if dropPersistence && n.persistent {
......@@ -389,14 +387,23 @@ func (n *Inode) removeRef(nlookup uint64, dropPersistence bool) (forgotten bool,
n.changeCounter++
}
n.bridge.mu.Lock()
if n.lookupCount == 0 {
forgotten = true
// Dropping the node from stableAttrs guarantees that no new references to this node are
// handed out to the kernel, hence we can also safely delete it from kernelNodeIds.
delete(n.bridge.stableAttrs, n.stableAttr)
delete(n.bridge.kernelNodeIds, n.nodeId)
}
n.bridge.mu.Unlock()
retry:
for {
lockme = append(lockme[:0], n)
parents = parents[:0]
nChange := n.changeCounter
live = n.lookupCount > 0 || len(n.children) > 0 || n.persistent
forgotten = n.lookupCount == 0
for p := range n.parents {
for _, p := range n.parents.all() {
parents = append(parents, p)
lockme = append(lockme, p.parent)
}
......@@ -415,20 +422,20 @@ retry:
}
for _, p := range parents {
if p.parent.children[p.name] != n {
// another node has replaced us already
continue
}
delete(p.parent.children, p.name)
p.parent.changeCounter++
}
n.parents = map[parentData]struct{}{}
n.parents.clear()
n.changeCounter++
if n.lookupCount != 0 {
panic("lookupCount changed")
log.Panicf("n%d %p lookupCount changed: %d", n.nodeId, n, n.lookupCount)
}
n.bridge.mu.Lock()
delete(n.bridge.nodes, n.stableAttr.Ino)
n.bridge.mu.Unlock()
unlockNodes(lockme...)
break
}
......@@ -463,7 +470,7 @@ retry:
parentCounter := n.changeCounter
if !ok {
n.children[name] = ch
ch.parents[parentData{name, n}] = struct{}{}
ch.parents.add(parentData{name, n})
n.changeCounter++
ch.changeCounter++
unlockNode2(n, ch)
......@@ -481,9 +488,9 @@ retry:
continue retry
}
delete(prev.parents, parentData{name, n})
prev.parents.delete(parentData{name, n})
n.children[name] = ch
ch.parents[parentData{name, n}] = struct{}{}
ch.parents.add(parentData{name, n})
n.changeCounter++
ch.changeCounter++
prev.changeCounter++
......@@ -509,10 +516,11 @@ func (n *Inode) Children() map[string]*Inode {
func (n *Inode) Parent() (string, *Inode) {
n.mu.Lock()
defer n.mu.Unlock()
for k := range n.parents {
return k.name, k.parent
p := n.parents.get()
if p == nil {
return "", nil
}
return "", nil
return p.name, p.parent
}
// RmAllChildren recursively drops a tree, forgetting all persistent
......@@ -558,15 +566,13 @@ retry:
lockNodes(lockme...)
if n.changeCounter != nChange {
unlockNodes(lockme...)
// could avoid unlocking and relocking n here.
n.mu.Lock()
continue retry
}
for _, nm := range names {
ch := n.children[nm]
delete(n.children, nm)
delete(ch.parents, parentData{nm, n})
ch.parents.delete(parentData{nm, n})
ch.changeCounter++
}
......@@ -617,7 +623,7 @@ retry:
if oldChild != nil {
delete(n.children, old)
delete(oldChild.parents, parentData{old, n})
oldChild.parents.delete(parentData{old, n})
n.changeCounter++
oldChild.changeCounter++
}
......@@ -626,7 +632,7 @@ retry:
// This can cause the child to be slated for
// removal; see below
delete(newParent.children, newName)
delete(destChild.parents, parentData{newName, newParent})
destChild.parents.delete(parentData{newName, newParent})
destChild.changeCounter++
newParent.changeCounter++
}
......@@ -635,7 +641,7 @@ retry:
newParent.children[newName] = oldChild
newParent.changeCounter++
oldChild.parents[parentData{newName, newParent}] = struct{}{}
oldChild.parents.add(parentData{newName, newParent})
oldChild.changeCounter++
}
......@@ -675,14 +681,14 @@ retry:
// Detach
if oldChild != nil {
delete(oldParent.children, oldName)
delete(oldChild.parents, parentData{oldName, oldParent})
oldChild.parents.delete(parentData{oldName, oldParent})
oldParent.changeCounter++
oldChild.changeCounter++
}
if destChild != nil {
delete(newParent.children, newName)
delete(destChild.parents, parentData{newName, newParent})
destChild.parents.delete(parentData{newName, newParent})
destChild.changeCounter++
newParent.changeCounter++
}
......@@ -692,15 +698,15 @@ retry:
newParent.children[newName] = oldChild
newParent.changeCounter++
oldChild.parents[parentData{newName, newParent}] = struct{}{}
oldChild.parents.add(parentData{newName, newParent})
oldChild.changeCounter++
}
if destChild != nil {
oldParent.children[oldName] = oldChild
oldParent.children[oldName] = destChild
oldParent.changeCounter++
destChild.parents[parentData{oldName, oldParent}] = struct{}{}
destChild.parents.add(parentData{oldName, oldParent})
destChild.changeCounter++
}
unlockNodes(oldParent, newParent, oldChild, destChild)
......@@ -712,7 +718,7 @@ retry:
// tuple should be invalidated. On next access, a LOOKUP operation
// will be started.
func (n *Inode) NotifyEntry(name string) syscall.Errno {
status := n.bridge.server.EntryNotify(n.stableAttr.Ino, name)
status := n.bridge.server.EntryNotify(n.nodeId, name)
return syscall.Errno(status)
}
......@@ -721,7 +727,7 @@ func (n *Inode) NotifyEntry(name string) syscall.Errno {
// to NotifyEntry, but also sends an event to inotify watchers.
func (n *Inode) NotifyDelete(name string, child *Inode) syscall.Errno {
// XXX arg ordering?
return syscall.Errno(n.bridge.server.DeleteNotify(n.stableAttr.Ino, child.stableAttr.Ino, name))
return syscall.Errno(n.bridge.server.DeleteNotify(n.nodeId, child.nodeId, name))
}
......@@ -729,16 +735,16 @@ func (n *Inode) NotifyDelete(name string, child *Inode) syscall.Errno {
// inode should be flushed from buffers.
func (n *Inode) NotifyContent(off, sz int64) syscall.Errno {
// XXX how does this work for directories?
return syscall.Errno(n.bridge.server.InodeNotify(n.stableAttr.Ino, off, sz))
return syscall.Errno(n.bridge.server.InodeNotify(n.nodeId, off, sz))
}
// WriteCache stores data in the kernel cache.
func (n *Inode) WriteCache(offset int64, data []byte) syscall.Errno {
return syscall.Errno(n.bridge.server.InodeNotifyStoreCache(n.stableAttr.Ino, offset, data))
return syscall.Errno(n.bridge.server.InodeNotifyStoreCache(n.nodeId, offset, data))
}
// ReadCache reads data from the kernel cache.
func (n *Inode) ReadCache(offset int64, dest []byte) (count int, errno syscall.Errno) {
c, s := n.bridge.server.InodeRetrieveCache(n.stableAttr.Ino, offset, dest)
c, s := n.bridge.server.InodeRetrieveCache(n.nodeId, offset, dest)
return c, syscall.Errno(s)
}
fs/inode_parents.go 0 → 100644
View file @ 43d5f40d
// Copyright 2021 the Go-FUSE Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package fs
// inodeParents stores zero or more parents of an Inode,
// remembering which one is the most recent.
//
// No internal locking: the caller is responsible for preventing
// concurrent access.
type inodeParents struct {
// newest is the most-recently add()'ed parent.
// nil when we don't have any parents.
newest *parentData
// other are parents in addition to the newest.
// nil or empty when we have <= 1 parents.
other map[parentData]struct{}
}
// add adds a parent to the store.
func (p *inodeParents) add(n parentData) {
// one and only parent
if p.newest == nil {
p.newest = &n
}
// already known as `newest`
if *p.newest == n {
return
}
// old `newest` gets displaced into `other`
if p.other == nil {
p.other = make(map[parentData]struct{})
}
p.other[*p.newest] = struct{}{}
// new parent becomes `newest` (possibly moving up from `other`)
delete(p.other, n)
p.newest = &n
}
// get returns the most recent parent
// or nil if there is no parent at all.
func (p *inodeParents) get() *parentData {
return p.newest
}
// all returns all known parents
// or nil if there is no parent at all.
func (p *inodeParents) all() []parentData {
count := p.count()
if count == 0 {
return nil
}
out := make([]parentData, 0, count)
out = append(out, *p.newest)
for i := range p.other {
out = append(out, i)
}
return out
}
func (p *inodeParents) delete(n parentData) {
// We have zero parents, so we can't delete any.
if p.newest == nil {
return
}
// If it's not the `newest` it must be in `other` (or nowhere).
if *p.newest != n {
delete(p.other, n)
return
}
// We want to delete `newest`, but there is no other to replace it.
if len(p.other) == 0 {
p.newest = nil
return
}
// Move random entry from `other` over `newest`.
var i parentData
for i = range p.other {
p.newest = &i
break
}
delete(p.other, i)
}
func (p *inodeParents) clear() {
p.newest = nil
p.other = nil
}
func (p *inodeParents) count() int {
if p.newest == nil {
return 0
}
return 1 + len(p.other)
}
type parentData struct {
name string
parent *Inode
}
fs/inode_parents_test.go 0 → 100644
View file @ 43d5f40d
package fs
import (
"testing"
)
func TestInodeParents(t *testing.T) {
var p inodeParents
var ino1, ino2, ino3 Inode
// empty store should be empty without panicing
if count := p.count(); count != 0 {
t.Error(count)
}
if p.all() != nil {
t.Error("empty store should return nil but did not")
}
// non-dupes should be stored
all := []parentData{
parentData{"foo", &ino1},
parentData{"foo2", &ino1},
parentData{"foo3", &ino1},
parentData{"foo", &ino2},
parentData{"foo", &ino3},
}
for i, v := range all {
p.add(v)
if count := p.count(); count != i+1 {
t.Errorf("want=%d have=%d", i+1, count)
}
last := p.get()
if *last != v {
t.Error("get did not give us last-known parent")
}
}
// adding dupes should not cause the count to increase, but
// must cause get() to return the most recently added dupe.
for _, v := range all {
p.add(v)
if count := p.count(); count != len(all) {
t.Errorf("want=%d have=%d", len(all), count)
}
last := p.get()
if *last != v {
t.Error("get did not give us last-known parent")
}
}
all2 := p.all()
if len(all) != len(all2) {
t.Errorf("want=%d have=%d", len(all), len(all2))
}
}
fs/inode_test.go 0 → 100644
View file @ 43d5f40d
package fs
import (
"syscall"
"testing"
)
func TestInodeIsDir(t *testing.T) {
cases := []struct {
mode uint32
dir bool
}{
{syscall.S_IFBLK, false},
{syscall.S_IFCHR, false},
{syscall.S_IFDIR, true},
{syscall.S_IFIFO, false},
{syscall.S_IFLNK, false},
{syscall.S_IFREG, false},
{syscall.S_IFSOCK, false},
}
var i Inode
for _, c := range cases {
i.stableAttr.Mode = c.mode
if i.IsDir() != c.dir {
t.Errorf("wrong result for case %#v", c)
}
}
}
fs/loopback.go
View file @ 43d5f40d
......@@ -13,39 +13,84 @@ import (
"github.com/hanwen/go-fuse/v2/fuse"
)
type loopbackRoot struct {
loopbackNode
// LoopbackRoot holds the parameters for creating a new loopback
// filesystem. Loopback filesystem delegate their operations to an
// underlying POSIX file system.
type LoopbackRoot struct {
// The path to the root of the underlying file system.
Path string
// The device on which the Path resides. This must be set if
// the underlying filesystem crosses file systems.
Dev uint64
// NewNode returns a new InodeEmbedder to be used to respond
// to a LOOKUP/CREATE/MKDIR/MKNOD opcode. If not set, use a
// LoopbackNode.
NewNode func(rootData *LoopbackRoot, parent *Inode, name string, st *syscall.Stat_t) InodeEmbedder
}
rootPath string
rootDev uint64
func (r *LoopbackRoot) newNode(parent *Inode, name string, st *syscall.Stat_t) InodeEmbedder {
if r.NewNode != nil {
return r.NewNode(r, parent, name, st)
}
return &LoopbackNode{
RootData: r,
}
}
func (r *LoopbackRoot) idFromStat(st *syscall.Stat_t) StableAttr {
// We compose an inode number by the underlying inode, and
// mixing in the device number. In traditional filesystems,
// the inode numbers are small. The device numbers are also
// small (typically 16 bit). Finally, we mask out the root
// device number of the root, so a loopback FS that does not
// encompass multiple mounts will reflect the inode numbers of
// the underlying filesystem
swapped := (uint64(st.Dev) << 32) | (uint64(st.Dev) >> 32)
swappedRootDev := (r.Dev << 32) | (r.Dev >> 32)
return StableAttr{
Mode: uint32(st.Mode),
Gen: 1,
// This should work well for traditional backing FSes,
// not so much for other go-fuse FS-es
Ino: (swapped ^ swappedRootDev) ^ st.Ino,
}
}
type loopbackNode struct {
// LoopbackNode is a filesystem node in a loopback file system. It is
// public so it can be used as a basis for other loopback based
// filesystems. See NewLoopbackFile or LoopbackRoot for more
// information.
type LoopbackNode struct {
Inode
// RootData points back to the root of the loopback filesystem.
RootData *LoopbackRoot
}
var _ = (NodeStatfser)((*loopbackNode)(nil))
var _ = (NodeStatfser)((*loopbackNode)(nil))
var _ = (NodeGetattrer)((*loopbackNode)(nil))
var _ = (NodeGetxattrer)((*loopbackNode)(nil))
var _ = (NodeSetxattrer)((*loopbackNode)(nil))
var _ = (NodeRemovexattrer)((*loopbackNode)(nil))
var _ = (NodeListxattrer)((*loopbackNode)(nil))
var _ = (NodeReadlinker)((*loopbackNode)(nil))
var _ = (NodeOpener)((*loopbackNode)(nil))
var _ = (NodeCopyFileRanger)((*loopbackNode)(nil))
var _ = (NodeLookuper)((*loopbackNode)(nil))
var _ = (NodeOpendirer)((*loopbackNode)(nil))
var _ = (NodeReaddirer)((*loopbackNode)(nil))
var _ = (NodeMkdirer)((*loopbackNode)(nil))
var _ = (NodeMknoder)((*loopbackNode)(nil))
var _ = (NodeLinker)((*loopbackNode)(nil))
var _ = (NodeSymlinker)((*loopbackNode)(nil))
var _ = (NodeUnlinker)((*loopbackNode)(nil))
var _ = (NodeRmdirer)((*loopbackNode)(nil))
var _ = (NodeRenamer)((*loopbackNode)(nil))
func (n *loopbackNode) Statfs(ctx context.Context, out *fuse.StatfsOut) syscall.Errno {
var _ = (NodeStatfser)((*LoopbackNode)(nil))
var _ = (NodeStatfser)((*LoopbackNode)(nil))
var _ = (NodeGetattrer)((*LoopbackNode)(nil))
var _ = (NodeGetxattrer)((*LoopbackNode)(nil))
var _ = (NodeSetxattrer)((*LoopbackNode)(nil))
var _ = (NodeRemovexattrer)((*LoopbackNode)(nil))
var _ = (NodeListxattrer)((*LoopbackNode)(nil))
var _ = (NodeReadlinker)((*LoopbackNode)(nil))
var _ = (NodeOpener)((*LoopbackNode)(nil))
var _ = (NodeCopyFileRanger)((*LoopbackNode)(nil))
var _ = (NodeLookuper)((*LoopbackNode)(nil))
var _ = (NodeOpendirer)((*LoopbackNode)(nil))
var _ = (NodeReaddirer)((*LoopbackNode)(nil))
var _ = (NodeMkdirer)((*LoopbackNode)(nil))
var _ = (NodeMknoder)((*LoopbackNode)(nil))
var _ = (NodeLinker)((*LoopbackNode)(nil))
var _ = (NodeSymlinker)((*LoopbackNode)(nil))
var _ = (NodeUnlinker)((*LoopbackNode)(nil))
var _ = (NodeRmdirer)((*LoopbackNode)(nil))
var _ = (NodeRenamer)((*LoopbackNode)(nil))
func (n *LoopbackNode) Statfs(ctx context.Context, out *fuse.StatfsOut) syscall.Errno {
s := syscall.Statfs_t{}
err := syscall.Statfs(n.path(), &s)
if err != nil {
......@@ -55,26 +100,14 @@ func (n *loopbackNode) Statfs(ctx context.Context, out *fuse.StatfsOut) syscall.
return OK
}
func (r *loopbackRoot) Getattr(ctx context.Context, f FileHandle, out *fuse.AttrOut) syscall.Errno {
st := syscall.Stat_t{}
err := syscall.Stat(r.rootPath, &st)
if err != nil {
return ToErrno(err)
}
out.FromStat(&st)
return OK
}
func (n *loopbackNode) root() *loopbackRoot {
return n.Root().Operations().(*loopbackRoot)
}
func (n *loopbackNode) path() string {
// path returns the full path to the file in the underlying file
// system.
func (n *LoopbackNode) path() string {
path := n.Path(n.Root())
return filepath.Join(n.root().rootPath, path)
return filepath.Join(n.RootData.Path, path)
}
func (n *loopbackNode) Lookup(ctx context.Context, name string, out *fuse.EntryOut) (*Inode, syscall.Errno) {
func (n *LoopbackNode) Lookup(ctx context.Context, name string, out *fuse.EntryOut) (*Inode, syscall.Errno) {
p := filepath.Join(n.path(), name)
st := syscall.Stat_t{}
......@@ -84,14 +117,14 @@ func (n *loopbackNode) Lookup(ctx context.Context, name string, out *fuse.EntryO
}
out.Attr.FromStat(&st)
node := &loopbackNode{}
ch := n.NewInode(ctx, node, n.root().idFromStat(&st))
node := n.RootData.newNode(n.EmbeddedInode(), name, &st)
ch := n.NewInode(ctx, node, n.RootData.idFromStat(&st))
return ch, 0
}
// preserveOwner sets uid and gid of `path` according to the caller information
// in `ctx`.
func (n *loopbackNode) preserveOwner(ctx context.Context, path string) error {
func (n *LoopbackNode) preserveOwner(ctx context.Context, path string) error {
if os.Getuid() != 0 {
return nil
}
......@@ -102,7 +135,7 @@ func (n *loopbackNode) preserveOwner(ctx context.Context, path string) error {
return syscall.Lchown(path, int(caller.Uid), int(caller.Gid))
}
func (n *loopbackNode) Mknod(ctx context.Context, name string, mode, rdev uint32, out *fuse.EntryOut) (*Inode, syscall.Errno) {
func (n *LoopbackNode) Mknod(ctx context.Context, name string, mode, rdev uint32, out *fuse.EntryOut) (*Inode, syscall.Errno) {
p := filepath.Join(n.path(), name)
err := syscall.Mknod(p, mode, int(rdev))
if err != nil {
......@@ -117,13 +150,13 @@ func (n *loopbackNode) Mknod(ctx context.Context, name string, mode, rdev uint32
out.Attr.FromStat(&st)
node := &loopbackNode{}
ch := n.NewInode(ctx, node, n.root().idFromStat(&st))
node := n.RootData.newNode(n.EmbeddedInode(), name, &st)
ch := n.NewInode(ctx, node, n.RootData.idFromStat(&st))
return ch, 0
}
func (n *loopbackNode) Mkdir(ctx context.Context, name string, mode uint32, out *fuse.EntryOut) (*Inode, syscall.Errno) {
func (n *LoopbackNode) Mkdir(ctx context.Context, name string, mode uint32, out *fuse.EntryOut) (*Inode, syscall.Errno) {
p := filepath.Join(n.path(), name)
err := os.Mkdir(p, os.FileMode(mode))
if err != nil {
......@@ -138,66 +171,39 @@ func (n *loopbackNode) Mkdir(ctx context.Context, name string, mode uint32, out
out.Attr.FromStat(&st)
node := &loopbackNode{}
ch := n.NewInode(ctx, node, n.root().idFromStat(&st))
node := n.RootData.newNode(n.EmbeddedInode(), name, &st)
ch := n.NewInode(ctx, node, n.RootData.idFromStat(&st))
return ch, 0
}
func (n *loopbackNode) Rmdir(ctx context.Context, name string) syscall.Errno {
func (n *LoopbackNode) Rmdir(ctx context.Context, name string) syscall.Errno {
p := filepath.Join(n.path(), name)
err := syscall.Rmdir(p)
return ToErrno(err)
}
func (n *loopbackNode) Unlink(ctx context.Context, name string) syscall.Errno {
func (n *LoopbackNode) Unlink(ctx context.Context, name string) syscall.Errno {
p := filepath.Join(n.path(), name)
err := syscall.Unlink(p)
return ToErrno(err)
}
func toLoopbackNode(op InodeEmbedder) *loopbackNode {
if r, ok := op.(*loopbackRoot); ok {
return &r.loopbackNode
}
return op.(*loopbackNode)
}
func (n *loopbackNode) Rename(ctx context.Context, name string, newParent InodeEmbedder, newName string, flags uint32) syscall.Errno {
newParentLoopback := toLoopbackNode(newParent)
func (n *LoopbackNode) Rename(ctx context.Context, name string, newParent InodeEmbedder, newName string, flags uint32) syscall.Errno {
if flags&RENAME_EXCHANGE != 0 {
return n.renameExchange(name, newParentLoopback, newName)
return n.renameExchange(name, newParent, newName)
}
p1 := filepath.Join(n.path(), name)
p2 := filepath.Join(newParentLoopback.path(), newName)
p2 := filepath.Join(n.RootData.Path, newParent.EmbeddedInode().Path(nil), newName)
err := syscall.Rename(p1, p2)
return ToErrno(err)
}
func (r *loopbackRoot) idFromStat(st *syscall.Stat_t) StableAttr {
// We compose an inode number by the underlying inode, and
// mixing in the device number. In traditional filesystems,
// the inode numbers are small. The device numbers are also
// small (typically 16 bit). Finally, we mask out the root
// device number of the root, so a loopback FS that does not
// encompass multiple mounts will reflect the inode numbers of
// the underlying filesystem
swapped := (uint64(st.Dev) << 32) | (uint64(st.Dev) >> 32)
swappedRootDev := (r.rootDev << 32) | (r.rootDev >> 32)
return StableAttr{
Mode: uint32(st.Mode),
Gen: 1,
// This should work well for traditional backing FSes,
// not so much for other go-fuse FS-es
Ino: (swapped ^ swappedRootDev) ^ st.Ino,
}
}
var _ = (NodeCreater)((*loopbackNode)(nil))
var _ = (NodeCreater)((*LoopbackNode)(nil))
func (n *loopbackNode) Create(ctx context.Context, name string, flags uint32, mode uint32, out *fuse.EntryOut) (inode *Inode, fh FileHandle, fuseFlags uint32, errno syscall.Errno) {
func (n *LoopbackNode) Create(ctx context.Context, name string, flags uint32, mode uint32, out *fuse.EntryOut) (inode *Inode, fh FileHandle, fuseFlags uint32, errno syscall.Errno) {
p := filepath.Join(n.path(), name)
flags = flags &^ syscall.O_APPEND
fd, err := syscall.Open(p, int(flags)|os.O_CREATE, mode)
......@@ -211,15 +217,15 @@ func (n *loopbackNode) Create(ctx context.Context, name string, flags uint32, mo
return nil, nil, 0, ToErrno(err)
}
node := &loopbackNode{}
ch := n.NewInode(ctx, node, n.root().idFromStat(&st))
node := n.RootData.newNode(n.EmbeddedInode(), name, &st)
ch := n.NewInode(ctx, node, n.RootData.idFromStat(&st))
lf := NewLoopbackFile(fd)
out.FromStat(&st)
return ch, lf, 0, 0
}
func (n *loopbackNode) Symlink(ctx context.Context, target, name string, out *fuse.EntryOut) (*Inode, syscall.Errno) {
func (n *LoopbackNode) Symlink(ctx context.Context, target, name string, out *fuse.EntryOut) (*Inode, syscall.Errno) {
p := filepath.Join(n.path(), name)
err := syscall.Symlink(target, p)
if err != nil {
......@@ -231,18 +237,17 @@ func (n *loopbackNode) Symlink(ctx context.Context, target, name string, out *fu
syscall.Unlink(p)
return nil, ToErrno(err)
}
node := &loopbackNode{}
ch := n.NewInode(ctx, node, n.root().idFromStat(&st))
node := n.RootData.newNode(n.EmbeddedInode(), name, &st)
ch := n.NewInode(ctx, node, n.RootData.idFromStat(&st))
out.Attr.FromStat(&st)
return ch, 0
}
func (n *loopbackNode) Link(ctx context.Context, target InodeEmbedder, name string, out *fuse.EntryOut) (*Inode, syscall.Errno) {
func (n *LoopbackNode) Link(ctx context.Context, target InodeEmbedder, name string, out *fuse.EntryOut) (*Inode, syscall.Errno) {
p := filepath.Join(n.path(), name)
targetNode := toLoopbackNode(target)
err := syscall.Link(targetNode.path(), p)
err := syscall.Link(filepath.Join(n.RootData.Path, target.EmbeddedInode().Path(nil)), p)
if err != nil {
return nil, ToErrno(err)
}
......@@ -251,14 +256,14 @@ func (n *loopbackNode) Link(ctx context.Context, target InodeEmbedder, name stri
syscall.Unlink(p)
return nil, ToErrno(err)
}
node := &loopbackNode{}
ch := n.NewInode(ctx, node, n.root().idFromStat(&st))
node := n.RootData.newNode(n.EmbeddedInode(), name, &st)
ch := n.NewInode(ctx, node, n.RootData.idFromStat(&st))
out.Attr.FromStat(&st)
return ch, 0
}
func (n *loopbackNode) Readlink(ctx context.Context) ([]byte, syscall.Errno) {
func (n *LoopbackNode) Readlink(ctx context.Context) ([]byte, syscall.Errno) {
p := n.path()
for l := 256; ; l *= 2 {
......@@ -274,7 +279,7 @@ func (n *loopbackNode) Readlink(ctx context.Context) ([]byte, syscall.Errno) {
}
}
func (n *loopbackNode) Open(ctx context.Context, flags uint32) (fh FileHandle, fuseFlags uint32, errno syscall.Errno) {
func (n *LoopbackNode) Open(ctx context.Context, flags uint32) (fh FileHandle, fuseFlags uint32, errno syscall.Errno) {
flags = flags &^ syscall.O_APPEND
p := n.path()
f, err := syscall.Open(p, int(flags), 0)
......@@ -285,7 +290,7 @@ func (n *loopbackNode) Open(ctx context.Context, flags uint32) (fh FileHandle, f
return lf, 0, 0
}
func (n *loopbackNode) Opendir(ctx context.Context) syscall.Errno {
func (n *LoopbackNode) Opendir(ctx context.Context) syscall.Errno {
fd, err := syscall.Open(n.path(), syscall.O_DIRECTORY, 0755)
if err != nil {
return ToErrno(err)
......@@ -294,19 +299,25 @@ func (n *loopbackNode) Opendir(ctx context.Context) syscall.Errno {
return OK
}
func (n *loopbackNode) Readdir(ctx context.Context) (DirStream, syscall.Errno) {
func (n *LoopbackNode) Readdir(ctx context.Context) (DirStream, syscall.Errno) {
return NewLoopbackDirStream(n.path())
}
func (n *loopbackNode) Getattr(ctx context.Context, f FileHandle, out *fuse.AttrOut) syscall.Errno {
func (n *LoopbackNode) Getattr(ctx context.Context, f FileHandle, out *fuse.AttrOut) syscall.Errno {
if f != nil {
return f.(FileGetattrer).Getattr(ctx, out)
}
p := n.path()
var err error
st := syscall.Stat_t{}
err = syscall.Lstat(p, &st)
if &n.Inode == n.Root() {
err = syscall.Stat(p, &st)
} else {
err = syscall.Lstat(p, &st)
}
if err != nil {
return ToErrno(err)
}
......@@ -314,9 +325,9 @@ func (n *loopbackNode) Getattr(ctx context.Context, f FileHandle, out *fuse.Attr
return OK
}
var _ = (NodeSetattrer)((*loopbackNode)(nil))
var _ = (NodeSetattrer)((*LoopbackNode)(nil))
func (n *loopbackNode) Setattr(ctx context.Context, f FileHandle, in *fuse.SetAttrIn, out *fuse.AttrOut) syscall.Errno {
func (n *LoopbackNode) Setattr(ctx context.Context, f FileHandle, in *fuse.SetAttrIn, out *fuse.AttrOut) syscall.Errno {
p := n.path()
fsa, ok := f.(FileSetattrer)
if ok && fsa != nil {
......@@ -390,16 +401,17 @@ func (n *loopbackNode) Setattr(ctx context.Context, f FileHandle, in *fuse.SetAt
// NewLoopbackRoot returns a root node for a loopback file system whose
// root is at the given root. This node implements all NodeXxxxer
// operations available.
func NewLoopbackRoot(root string) (InodeEmbedder, error) {
func NewLoopbackRoot(rootPath string) (InodeEmbedder, error) {
var st syscall.Stat_t
err := syscall.Stat(root, &st)
err := syscall.Stat(rootPath, &st)
if err != nil {
return nil, err
}
n := &loopbackRoot{
rootPath: root,
rootDev: uint64(st.Dev),
root := &LoopbackRoot{
Path: rootPath,
Dev: uint64(st.Dev),
}
return n, nil
return root.newNode(nil, "", &st), nil
}
fs/loopback_darwin.go
View file @ 43d5f40d
......@@ -16,23 +16,23 @@ import (
"github.com/hanwen/go-fuse/v2/internal/utimens"
)
func (n *loopbackNode) Getxattr(ctx context.Context, attr string, dest []byte) (uint32, syscall.Errno) {
func (n *LoopbackNode) Getxattr(ctx context.Context, attr string, dest []byte) (uint32, syscall.Errno) {
return 0, syscall.ENOSYS
}
func (n *loopbackNode) Setxattr(ctx context.Context, attr string, data []byte, flags uint32) syscall.Errno {
func (n *LoopbackNode) Setxattr(ctx context.Context, attr string, data []byte, flags uint32) syscall.Errno {
return syscall.ENOSYS
}
func (n *loopbackNode) Removexattr(ctx context.Context, attr string) syscall.Errno {
func (n *LoopbackNode) Removexattr(ctx context.Context, attr string) syscall.Errno {
return syscall.ENOSYS
}
func (n *loopbackNode) Listxattr(ctx context.Context, dest []byte) (uint32, syscall.Errno) {
func (n *LoopbackNode) Listxattr(ctx context.Context, dest []byte) (uint32, syscall.Errno) {
return 0, syscall.ENOSYS
}
func (n *loopbackNode) renameExchange(name string, newparent *loopbackNode, newName string) syscall.Errno {
func (n *LoopbackNode) renameExchange(name string, newparent InodeEmbedder, newName string) syscall.Errno {
return syscall.ENOSYS
}
......@@ -111,7 +111,7 @@ func (f *loopbackFile) utimens(a *time.Time, m *time.Time) syscall.Errno {
return ToErrno(err)
}
func (n *loopbackNode) CopyFileRange(ctx context.Context, fhIn FileHandle,
func (n *LoopbackNode) CopyFileRange(ctx context.Context, fhIn FileHandle,
offIn uint64, out *Inode, fhOut FileHandle, offOut uint64,
len uint64, flags uint64) (uint32, syscall.Errno) {
return 0, syscall.ENOSYS
......
fs/loopback_linux.go
View file @ 43d5f40d
......@@ -8,38 +8,40 @@ package fs
import (
"context"
"path/filepath"
"syscall"
"golang.org/x/sys/unix"
)
func (n *loopbackNode) Getxattr(ctx context.Context, attr string, dest []byte) (uint32, syscall.Errno) {
func (n *LoopbackNode) Getxattr(ctx context.Context, attr string, dest []byte) (uint32, syscall.Errno) {
sz, err := unix.Lgetxattr(n.path(), attr, dest)
return uint32(sz), ToErrno(err)
}
func (n *loopbackNode) Setxattr(ctx context.Context, attr string, data []byte, flags uint32) syscall.Errno {
func (n *LoopbackNode) Setxattr(ctx context.Context, attr string, data []byte, flags uint32) syscall.Errno {
err := unix.Lsetxattr(n.path(), attr, data, int(flags))
return ToErrno(err)
}
func (n *loopbackNode) Removexattr(ctx context.Context, attr string) syscall.Errno {
func (n *LoopbackNode) Removexattr(ctx context.Context, attr string) syscall.Errno {
err := unix.Lremovexattr(n.path(), attr)
return ToErrno(err)
}
func (n *loopbackNode) Listxattr(ctx context.Context, dest []byte) (uint32, syscall.Errno) {
func (n *LoopbackNode) Listxattr(ctx context.Context, dest []byte) (uint32, syscall.Errno) {
sz, err := unix.Llistxattr(n.path(), dest)
return uint32(sz), ToErrno(err)
}
func (n *loopbackNode) renameExchange(name string, newparent *loopbackNode, newName string) syscall.Errno {
func (n *LoopbackNode) renameExchange(name string, newparent InodeEmbedder, newName string) syscall.Errno {
fd1, err := syscall.Open(n.path(), syscall.O_DIRECTORY, 0)
if err != nil {
return ToErrno(err)
}
defer syscall.Close(fd1)
fd2, err := syscall.Open(newparent.path(), syscall.O_DIRECTORY, 0)
p2 := filepath.Join(n.RootData.Path, newparent.EmbeddedInode().Path(nil))
fd2, err := syscall.Open(p2, syscall.O_DIRECTORY, 0)
defer syscall.Close(fd2)
if err != nil {
return ToErrno(err)
......@@ -52,22 +54,22 @@ func (n *loopbackNode) renameExchange(name string, newparent *loopbackNode, newN
// Double check that nodes didn't change from under us.
inode := &n.Inode
if inode.Root() != inode && inode.StableAttr().Ino != n.root().idFromStat(&st).Ino {
if inode.Root() != inode && inode.StableAttr().Ino != n.RootData.idFromStat(&st).Ino {
return syscall.EBUSY
}
if err := syscall.Fstat(fd2, &st); err != nil {
return ToErrno(err)
}
newinode := &newparent.Inode
if newinode.Root() != newinode && newinode.StableAttr().Ino != n.root().idFromStat(&st).Ino {
newinode := newparent.EmbeddedInode()
if newinode.Root() != newinode && newinode.StableAttr().Ino != n.RootData.idFromStat(&st).Ino {
return syscall.EBUSY
}
return ToErrno(unix.Renameat2(fd1, name, fd2, newName, unix.RENAME_EXCHANGE))
}
func (n *loopbackNode) CopyFileRange(ctx context.Context, fhIn FileHandle,
func (n *LoopbackNode) CopyFileRange(ctx context.Context, fhIn FileHandle,
offIn uint64, out *Inode, fhOut FileHandle, offOut uint64,
len uint64, flags uint64) (uint32, syscall.Errno) {
lfIn, ok := fhIn.(*loopbackFile)
......
fs/loopback_linux_test.go
View file @ 43d5f40d
......@@ -75,6 +75,22 @@ func TestRenameExchange(t *testing.T) {
if !reflect.DeepEqual(after2, before1) {
t.Errorf("after2, before1: %#v, %#v", after2, before1)
}
root := tc.loopback.EmbeddedInode().Root()
ino1 := root.GetChild("file")
if ino1 == nil {
t.Fatalf("root.GetChild(%q): null inode", "file")
}
ino2 := root.GetChild("dir").GetChild("file")
if ino2 == nil {
t.Fatalf("dir.GetChild(%q): null inode", "file")
}
if ino1.StableAttr().Ino != after1.Ino {
t.Errorf("got inode %d for %q, want %d", ino1.StableAttr().Ino, "file", after1.Ino)
}
if ino2.StableAttr().Ino != after2.Ino {
t.Errorf("got inode %d for %q want %d", ino2.StableAttr().Ino, "dir/file", after2.Ino)
}
}
func TestRenameNoOverwrite(t *testing.T) {
......
fs/rmchild_test.go 0 → 100644
View file @ 43d5f40d
// Copyright 2019 the Go-FUSE Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package fs
import (
"context"
"fmt"
"sync"
"testing"
"github.com/hanwen/go-fuse/v2/fuse"
)
func TestRmChildParallel(t *testing.T) {
want := "hello"
root := &Inode{}
_, _, clean := testMount(t, root, &Options{
FirstAutomaticIno: 1,
OnAdd: func(ctx context.Context) {
n := root.EmbeddedInode()
var wg sync.WaitGroup
var nms []string
for i := 0; i < 100; i++ {
nms = append(nms, fmt.Sprint(i))
}
for _, nm := range nms {
wg.Add(1)
go func(nm string) {
ch := n.NewPersistentInode(
ctx,
&MemRegularFile{
Data: []byte(want),
Attr: fuse.Attr{
Mode: 0464,
},
},
StableAttr{})
n.AddChild(nm, ch, false)
wg.Done()
}(nm)
}
for _, nm := range nms {
wg.Add(1)
go func(nm string) {
n.RmChild(nm)
wg.Done()
}(nm)
}
wg.Wait()
},
})
defer clean()
}
fs/simple_test.go
View file @ 43d5f40d
......@@ -5,12 +5,19 @@
package fs
import (
"context"
"fmt"
"io/ioutil"
"log"
"os"
"os/exec"
"path/filepath"
"reflect"
"runtime"
"sort"
"strings"
"sync"
"sync/atomic"
"syscall"
"testing"
"time"
......@@ -97,6 +104,7 @@ func newTestCase(t *testing.T, opts *testOptions) *testCase {
tc.rawFS = NewNodeFS(tc.loopback, &Options{
EntryTimeout: entryDT,
AttrTimeout: attrDT,
Logger: log.New(os.Stderr, "", 0),
})
mOpts := &fuse.MountOptions{}
......@@ -349,6 +357,24 @@ func TestMknod(t *testing.T) {
}
}
func TestMknodNotSupported(t *testing.T) {
mountPoint := testutil.TempDir()
defer os.Remove(mountPoint)
server, err := Mount(mountPoint, &Inode{}, nil)
if err != nil {
t.Fatalf("cannot mount: %v", err)
}
defer server.Unmount()
name := filepath.Join(mountPoint, "foo")
if got, want := syscall.Mknod(name, syscall.S_IFREG|0755, (8<<8)|0), syscall.ENOTSUP; got != want {
t.Fatalf("mknod: got %v, want %v", got, want)
}
}
func TestPosix(t *testing.T) {
noisy := map[string]bool{
"ParallelFileOpen": true,
......@@ -392,6 +418,264 @@ func TestOpenDirectIO(t *testing.T) {
posixtest.DirectIO(t, tc.mntDir)
}
// TestFsstress is loosely modeled after xfstest's fsstress. It performs rapid
// parallel removes / creates / readdirs. Coupled with inode reuse, this test
// used to deadlock go-fuse quite quickly.
//
// Note: Run as
//
// TMPDIR=/var/tmp go test -run TestFsstress
//
// to make sure the backing filesystem is ext4. /tmp is tmpfs on modern Linux
// distributions, and tmpfs does not reuse inode numbers, hiding the problem.
func TestFsstress(t *testing.T) {
tc := newTestCase(t, &testOptions{suppressDebug: true, attrCache: true, entryCache: true})
defer tc.Clean()
{
old := runtime.GOMAXPROCS(100)
defer runtime.GOMAXPROCS(old)
}
const concurrency = 10
var wg sync.WaitGroup
ctx, cancel := context.WithCancel(context.Background())
// operations taking 1 path argument
ops1 := map[string]func(string) error{
"mkdir": func(p string) error { return syscall.Mkdir(p, 0700) },
"rmdir": func(p string) error { return syscall.Rmdir(p) },
"mknod_reg": func(p string) error { return syscall.Mknod(p, 0700|syscall.S_IFREG, 0) },
"remove": os.Remove,
"unlink": syscall.Unlink,
"mknod_sock": func(p string) error { return syscall.Mknod(p, 0700|syscall.S_IFSOCK, 0) },
"mknod_fifo": func(p string) error { return syscall.Mknod(p, 0700|syscall.S_IFIFO, 0) },
"mkfifo": func(p string) error { return syscall.Mkfifo(p, 0700) },
"symlink": func(p string) error { return syscall.Symlink("foo", p) },
"creat": func(p string) error {
fd, err := syscall.Open(p, syscall.O_CREAT|syscall.O_EXCL, 0700)
if err == nil {
syscall.Close(fd)
}
return err
},
}
// operations taking 2 path arguments
ops2 := map[string]func(string, string) error{
"rename": syscall.Rename,
"link": syscall.Link,
}
type opStats struct {
ok *int64
fail *int64
hung *int64
}
stats := make(map[string]opStats)
// pathN() returns something like /var/tmp/TestFsstress/TestFsstress.4
pathN := func(n int) string {
return fmt.Sprintf("%s/%s.%d", tc.mntDir, t.Name(), n)
}
opLoop := func(k string, n int) {
defer wg.Done()
op := ops1[k]
for {
p := pathN(1)
atomic.AddInt64(stats[k].hung, 1)
err := op(p)
atomic.AddInt64(stats[k].hung, -1)
if err != nil {
atomic.AddInt64(stats[k].fail, 1)
} else {
atomic.AddInt64(stats[k].ok, 1)
}
select {
case <-ctx.Done():
return
default:
}
}
}
op2Loop := func(k string, n int) {
defer wg.Done()
op := ops2[k]
n2 := (n + 1) % concurrency
for {
p1 := pathN(n)
p2 := pathN(n2)
atomic.AddInt64(stats[k].hung, 1)
err := op(p1, p2)
atomic.AddInt64(stats[k].hung, -1)
if err != nil {
atomic.AddInt64(stats[k].fail, 1)
} else {
atomic.AddInt64(stats[k].ok, 1)
}
select {
case <-ctx.Done():
return
default:
}
}
}
readdirLoop := func(k string) {
defer wg.Done()
for {
atomic.AddInt64(stats[k].hung, 1)
f, err := os.Open(tc.mntDir)
if err != nil {
panic(err)
}
_, err = f.Readdir(0)
if err != nil {
atomic.AddInt64(stats[k].fail, 1)
} else {
atomic.AddInt64(stats[k].ok, 1)
}
f.Close()
atomic.AddInt64(stats[k].hung, -1)
select {
case <-ctx.Done():
return
default:
}
}
}
// prepare stats map
var allOps []string
for k := range ops1 {
allOps = append(allOps, k)
}
for k := range ops2 {
allOps = append(allOps, k)
}
allOps = append(allOps, "readdir")
for _, k := range allOps {
var i1, i2, i3 int64
stats[k] = opStats{ok: &i1, fail: &i2, hung: &i3}
}
// spawn worker goroutines
for i := 0; i < concurrency; i++ {
for k := range ops1 {
wg.Add(1)
go opLoop(k, i)
}
for k := range ops2 {
wg.Add(1)
go op2Loop(k, i)
}
}
{
k := "readdir"
wg.Add(1)
go readdirLoop(k)
}
// spawn ls loop
//
// An external "ls" loop has a destructive effect that I am unable to
// reproduce through in-process operations.
if strings.ContainsAny(tc.mntDir, "'\\") {
// But let's not enable shell injection.
log.Panicf("shell injection attempt? mntDir=%q", tc.mntDir)
}
// --color=always enables xattr lookups for extra stress
cmd := exec.Command("bash", "-c", "while true ; do ls -l --color=always '"+tc.mntDir+"'; done")
err := cmd.Start()
if err != nil {
t.Fatal(err)
}
defer cmd.Process.Kill()
// Run the test for 1 second. If it deadlocks, it usually does within 20ms.
time.Sleep(1 * time.Second)
cancel()
// waitTimeout waits for the waitgroup for the specified max timeout.
// Returns true if waiting timed out.
waitTimeout := func(wg *sync.WaitGroup, timeout time.Duration) bool {
c := make(chan struct{})
go func() {
defer close(c)
wg.Wait()
}()
select {
case <-c:
return false // completed normally
case <-time.After(timeout):
return true // timed out
}
}
if waitTimeout(&wg, time.Second) {
t.Errorf("timeout waiting for goroutines to exit (deadlocked?)")
}
// Print operation statistics
var keys []string
for k := range stats {
keys = append(keys, k)
}
sort.Strings(keys)
t.Logf("Operation statistics:")
for _, k := range keys {
v := stats[k]
t.Logf("%10s: %5d ok, %6d fail, %2d hung", k, *v.ok, *v.fail, *v.hung)
}
}
// TestStaleHardlinks creates a lot of hard links and deletes them again
// behind the back of the loopback fs. Then opens the original file.
//
// Fails at the moment. Core of the problem:
//
// 18:41:50.796468 rx 136: LOOKUP n1 ["link0"] 6b
// 18:41:50.796489 tx 136: OK, {n2 g1 tE=0s tA=0s {M0100600 SZ=0 L=1 1026:1026 B0*4096 i0:269663 A 1616348510.793212 M 1616348510.793212 C 1616348510.795212}}
// 18:41:50.796535 rx 138: OPEN n2 {O_RDONLY,0x8000}
// 18:41:50.796557 tx 138: 2=no such file or directory, {Fh 0 }
func TestStaleHardlinks(t *testing.T) {
// Disable all caches we can disable
tc := newTestCase(t, &testOptions{attrCache: false, entryCache: false})
defer tc.Clean()
// "link0" is original file
link0 := tc.mntDir + "/link0"
if fd, err := syscall.Creat(link0, 0600); err != nil {
t.Fatal(err)
} else {
syscall.Close(fd)
}
// Create hardlinks via mntDir
for i := 1; i < 20; i++ {
linki := fmt.Sprintf(tc.mntDir+"/link%d", i)
if err := syscall.Link(link0, linki); err != nil {
t.Fatal(err)
}
}
// Delete hardlinks via origDir (behind loopback fs's back)
for i := 1; i < 20; i++ {
linki := fmt.Sprintf(tc.origDir+"/link%d", i)
if err := syscall.Unlink(linki); err != nil {
t.Fatal(err)
}
}
// Try to open link0 via mntDir
fd, err := syscall.Open(link0, syscall.O_RDONLY, 0)
if err != nil {
t.Error(err)
} else {
syscall.Close(fd)
}
}
func init() {
syscall.Umask(0)
}
fs/windows_example_test.go 0 → 100644
View file @ 43d5f40d
// Copyright 2020 the Go-FUSE Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package fs_test
import (
"context"
"fmt"
"log"
"sync"
"syscall"
"time"
"github.com/hanwen/go-fuse/v2/fs"
"github.com/hanwen/go-fuse/v2/fuse"
)
// WindowsNode emulates Windows FS semantics, which forbids deleting open files.
type WindowsNode struct {
// WindowsNode inherits most functionality from LoopbackNode.
fs.LoopbackNode
mu sync.Mutex
openCount int
}
var _ = (fs.NodeOpener)((*WindowsNode)(nil))
func (n *WindowsNode) Open(ctx context.Context, flags uint32) (fs.FileHandle, uint32, syscall.Errno) {
fh, flags, errno := n.LoopbackNode.Open(ctx, flags)
if errno == 0 {
n.mu.Lock()
defer n.mu.Unlock()
n.openCount++
}
return fh, flags, errno
}
var _ = (fs.NodeCreater)((*WindowsNode)(nil))
func (n *WindowsNode) Create(ctx context.Context, name string, flags uint32, mode uint32, out *fuse.EntryOut) (*fs.Inode, fs.FileHandle, uint32, syscall.Errno) {
inode, fh, flags, errno := n.LoopbackNode.Create(ctx, name, flags, mode, out)
if errno == 0 {
wn := inode.Operations().(*WindowsNode)
wn.openCount++
}
return inode, fh, flags, errno
}
var _ = (fs.NodeReleaser)((*WindowsNode)(nil))
// Release decreases the open count. The kernel doesn't wait with
// returning from close(), so if the caller is too quick to
// unlink/rename after calling close(), this may still trigger EBUSY.
func (n *WindowsNode) Release(ctx context.Context, f fs.FileHandle) syscall.Errno {
n.mu.Lock()
defer n.mu.Unlock()
n.openCount--
if fr, ok := f.(fs.FileReleaser); ok {
return fr.Release(ctx)
}
return 0
}
func isBusy(parent *fs.Inode, name string) bool {
if ch := parent.GetChild(name); ch != nil {
if wn, ok := ch.Operations().(*WindowsNode); ok {
wn.mu.Lock()
defer wn.mu.Unlock()
if wn.openCount > 0 {
return true
}
}
}
return false
}
var _ = (fs.NodeUnlinker)((*WindowsNode)(nil))
func (n *WindowsNode) Unlink(ctx context.Context, name string) syscall.Errno {
if isBusy(n.EmbeddedInode(), name) {
return syscall.EBUSY
}
return n.LoopbackNode.Unlink(ctx, name)
}
func newWindowsNode(rootData *fs.LoopbackRoot, parent *fs.Inode, name string, st *syscall.Stat_t) fs.InodeEmbedder {
n := &WindowsNode{
LoopbackNode: fs.LoopbackNode{
RootData: rootData,
},
}
return n
}
// ExampleLoopbackReuse shows how to build a file system on top of the
// loopback file system.
func Example_loopbackReuse() {
mntDir := "/tmp/mnt"
origDir := "/tmp/orig"
rootData := &fs.LoopbackRoot{
NewNode: newWindowsNode,
Path: origDir,
}
sec := time.Second
opts := &fs.Options{
AttrTimeout: &sec,
EntryTimeout: &sec,
}
server, err := fs.Mount(mntDir, newWindowsNode(rootData, nil, "", nil), opts)
if err != nil {
log.Fatalf("Mount fail: %v\n", err)
}
fmt.Printf("files under %s cannot be deleted if they are opened", mntDir)
server.Wait()
}
fs/windows_test.go 0 → 100644
View file @ 43d5f40d
// Copyright 2020 the Go-FUSE Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package fs_test
import (
"bytes"
"io/ioutil"
"os"
"syscall"
"testing"
"time"
"github.com/hanwen/go-fuse/v2/fs"
"github.com/hanwen/go-fuse/v2/internal/testutil"
)
func TestWindowsEmulations(t *testing.T) {
mntDir, err := ioutil.TempDir("", "ZipFS")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(mntDir)
origDir, err := ioutil.TempDir("", "ZipFS")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(origDir)
rootData := &fs.LoopbackRoot{
NewNode: newWindowsNode,
Path: origDir,
}
opts := fs.Options{}
opts.Debug = testutil.VerboseTest()
server, err := fs.Mount(mntDir, newWindowsNode(rootData, nil, "", nil), &opts)
if err != nil {
t.Fatal(err)
}
defer server.Unmount()
data := []byte("hello")
nm := mntDir + "/file"
if err := ioutil.WriteFile(nm, data, 0644); err != nil {
t.Fatal(err)
}
if got, err := ioutil.ReadFile(nm); err != nil {
t.Fatal(err)
} else if bytes.Compare(got, data) != 0 {
t.Fatalf("got %q want %q", got, data)
}
f, err := os.Open(nm)
if err != nil {
t.Fatal(err)
}
if err := syscall.Unlink(nm); err == nil {
t.Fatal("Unlink should have failed")
}
f.Close()
// Ugh - it may take a while for the RELEASE to be processed.
time.Sleep(10 * time.Millisecond)
if err := syscall.Unlink(nm); err != nil {
t.Fatalf("Unlink: %v", err)
}
}
fuse/api.go
View file @ 43d5f40d
......@@ -67,21 +67,59 @@
// Typically, each call of the API happens in its own
// goroutine, so take care to make the file system thread-safe.
//
// Be careful when you access the FUSE mount from the same process. An access can
// tie up two OS threads (one on the request side and one on the FUSE server side).
// This can deadlock if there is no free thread to handle the FUSE server side.
// Run your program with GOMAXPROCS=1 to make the problem easier to reproduce,
// see https://github.com/hanwen/go-fuse/issues/261 for an example of that
// problem.
//
// Higher level interfaces
//
// As said above this packages provides way to implement filesystems in terms of
// raw FUSE protocol. Additionally packages nodefs and pathfs provide ways to
// implement filesystem at higher levels:
// raw FUSE protocol.
//
// Package github.com/hanwen/go-fuse/fuse/nodefs provides way to implement
// filesystems in terms of inodes. This resembles kernel's idea of what a
// filesystem looks like.
// Package github.com/hanwen/go-fuse/v2/fs provides way to implement
// filesystems in terms of paths and/or inodes.
//
// Package github.com/hanwen/go-fuse/fuse/pathfs provides way to implement
// filesystems in terms of path names. Working with path names is somewhat
// easier compared to inodes, however renames can be racy. Do not use pathfs if
// you care about correctness.
// Mount styles
//
// The NewServer() handles mounting the filesystem, which
// involves opening `/dev/fuse` and calling the
// `mount(2)` syscall. The latter needs root permissions.
// This is handled in one of three ways:
//
// 1) go-fuse opens `/dev/fuse` and executes the `fusermount`
// setuid-root helper to call `mount(2)` for us. This is the default.
// Does not need root permissions but needs `fusermount` installed.
//
// 2) If `MountOptions.DirectMount` is set, go-fuse calls `mount(2)` itself.
// Needs root permissions, but works without `fusermount`.
//
// 3) If `mountPoint` has the magic `/dev/fd/N` syntax, it means that that a
// privileged parent process:
//
// * Opened /dev/fuse
//
// * Called mount(2) on a real mountpoint directory that we don't know about
//
// * Inherited the fd to /dev/fuse to us
//
// * Informs us about the fd number via /dev/fd/N
//
// This magic syntax originates from libfuse [1] and allows the FUSE server to
// run without any privileges and without needing `fusermount`, as the parent
// process performs all privileged operations.
//
// The "privileged parent" is usually a container manager like Singularity [2],
// but for testing, it can also be the `mount.fuse3` helper with the
// `drop_privileges,setuid=$USER` flags. Example below for gocryptfs:
//
// $ sudo mount.fuse3 "/usr/local/bin/gocryptfs#/tmp/cipher" /tmp/mnt -o drop_privileges,setuid=$USER
//
// [1] https://github.com/libfuse/libfuse/commit/64e11073b9347fcf9c6d1eea143763ba9e946f70
//
// [2] https://sylabs.io/guides/3.7/user-guide/bind_paths_and_mounts.html#fuse-mounts
package fuse
// Types for users to implement.
......@@ -159,6 +197,23 @@ type MountOptions struct {
// The filesystem is fully responsible for invalidating data cache.
ExplicitDataCacheControl bool
// SyncRead is off by default, which means that go-fuse enable the
// FUSE_CAP_ASYNC_READ capability.
// The kernel then submits multiple concurrent reads to service
// userspace requests and kernel readahead.
//
// Setting SyncRead disables the FUSE_CAP_ASYNC_READ capability.
// The kernel then only sends one read request per file handle at a time,
// and orders the requests by offset.
//
// This is useful if reading out of order or concurrently is expensive for
// (example: Amazon Cloud Drive).
//
// See the comment to FUSE_CAP_ASYNC_READ in
// https://github.com/libfuse/libfuse/blob/master/include/fuse_common.h
// for more details.
SyncRead bool
// If set, fuse will first attempt to use syscall.Mount instead of
// fusermount to mount the filesystem. This will not update /etc/mtab
// but might be needed if fusermount is not available.
......@@ -167,6 +222,18 @@ type MountOptions struct {
// Options passed to syscall.Mount, the default value used by fusermount
// is syscall.MS_NOSUID|syscall.MS_NODEV
DirectMountFlags uintptr
// EnableAcls enables kernel ACL support.
//
// See the comments to FUSE_CAP_POSIX_ACL
// in https://github.com/libfuse/libfuse/blob/master/include/fuse_common.h
// for details.
EnableAcl bool
// Disable ReadDirPlus capability so ReadDir is used instead. Simple
// directory queries (i.e. 'ls' without '-l') can be faster with
// ReadDir, as no per-file stat calls are needed
DisableReadDirPlus bool
}
// RawFileSystem is an interface close to the FUSE wire protocol.
......
fuse/attr_darwin.go
View file @ 43d5f40d
......@@ -23,4 +23,5 @@ func (a *Attr) FromStat(s *syscall.Stat_t) {
a.Uid = uint32(s.Uid)
a.Gid = uint32(s.Gid)
a.Rdev = uint32(s.Rdev)
a.Blksize = uint32(s.Blksize)
}
fuse/constants.go
View file @ 43d5f40d
......@@ -18,6 +18,9 @@ const (
FUSE_LK_FLOCK = (1 << 0)
FUSE_RELEASE_FLUSH = (1 << 0)
FUSE_RELEASE_FLOCK_UNLOCK = (1 << 1)
FUSE_IOCTL_MAX_IOV = 256
FUSE_POLL_SCHEDULE_NOTIFY = (1 << 0)
......
fuse/mount_darwin.go
View file @ 43d5f40d
......@@ -9,89 +9,125 @@ import (
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"syscall"
"unsafe"
)
func openFUSEDevice() (*os.File, error) {
fs, err := filepath.Glob("/dev/osxfuse*")
func unixgramSocketpair() (l, r *os.File, err error) {
fd, err := syscall.Socketpair(syscall.AF_UNIX, syscall.SOCK_STREAM, 0)
if err != nil {
return nil, err
return nil, nil, os.NewSyscallError("socketpair",
err.(syscall.Errno))
}
if len(fs) == 0 {
bin := oldLoadBin
if _, err := os.Stat(newLoadBin); err == nil {
bin = newLoadBin
}
cmd := exec.Command(bin)
if err := cmd.Run(); err != nil {
return nil, err
}
fs, err = filepath.Glob("/dev/osxfuse*")
if err != nil {
return nil, err
}
}
for _, fn := range fs {
f, err := os.OpenFile(fn, os.O_RDWR, 0)
if err != nil {
continue
}
return f, nil
}
return nil, fmt.Errorf("all FUSE devices busy")
l = os.NewFile(uintptr(fd[0]), "socketpair-half1")
r = os.NewFile(uintptr(fd[1]), "socketpair-half2")
return
}
const oldLoadBin = "/Library/Filesystems/osxfusefs.fs/Support/load_osxfusefs"
const newLoadBin = "/Library/Filesystems/osxfuse.fs/Contents/Resources/load_osxfuse"
const oldMountBin = "/Library/Filesystems/osxfusefs.fs/Support/mount_osxfusefs"
const newMountBin = "/Library/Filesystems/osxfuse.fs/Contents/Resources/mount_osxfuse"
// Create a FUSE FS on the specified mount point. The returned
// mount point is always absolute.
func mount(mountPoint string, opts *MountOptions, ready chan<- error) (fd int, err error) {
f, err := openFUSEDevice()
local, remote, err := unixgramSocketpair()
if err != nil {
return 0, err
return
}
bin := oldMountBin
if _, err := os.Stat(newMountBin); err == nil {
bin = newMountBin
defer local.Close()
defer remote.Close()
bin, err := fusermountBinary()
if err != nil {
return 0, err
}
cmd := exec.Command(bin, "-o", strings.Join(opts.optionsStrings(), ","), "-o", fmt.Sprintf("iosize=%d", opts.MaxWrite), "3", mountPoint)
cmd.ExtraFiles = []*os.File{f}
cmd.Env = append(os.Environ(), "MOUNT_FUSEFS_CALL_BY_LIB=", "MOUNT_OSXFUSE_CALL_BY_LIB=",
"MOUNT_OSXFUSE_DAEMON_PATH="+os.Args[0],
"MOUNT_FUSEFS_DAEMON_PATH="+os.Args[0])
cmd := exec.Command(bin,
"-o", strings.Join(opts.optionsStrings(), ","),
"-o", fmt.Sprintf("iosize=%d", opts.MaxWrite),
mountPoint)
cmd.ExtraFiles = []*os.File{remote} // fd would be (index + 3)
cmd.Env = append(os.Environ(),
"_FUSE_CALL_BY_LIB=",
"_FUSE_DAEMON_PATH="+os.Args[0],
"_FUSE_COMMFD=3",
"_FUSE_COMMVERS=2",
"MOUNT_OSXFUSE_CALL_BY_LIB=",
"MOUNT_OSXFUSE_DAEMON_PATH="+os.Args[0])
var out, errOut bytes.Buffer
cmd.Stdout = &out
cmd.Stderr = &errOut
if err := cmd.Start(); err != nil {
f.Close()
return 0, err
if err = cmd.Start(); err != nil {
return
}
fd, err = getConnection(local)
if err != nil {
return -1, err
}
go func() {
err := cmd.Wait()
if err != nil {
err = fmt.Errorf("mount_osxfusefs failed: %v. Stderr: %s, Stdout: %s", err, errOut.String(), out.String())
// wait inside a goroutine or otherwise it would block forever for unknown reasons
if err := cmd.Wait(); err != nil {
err = fmt.Errorf("mount_osxfusefs failed: %v. Stderr: %s, Stdout: %s",
err, errOut.String(), out.String())
}
ready <- err
close(ready)
}()
// The finalizer for f will close its fd so we return a dup.
defer f.Close()
return syscall.Dup(int(f.Fd()))
// golang sets CLOEXEC on file descriptors when they are
// acquired through normal operations (e.g. open).
// Buf for fd, we have to set CLOEXEC manually
syscall.CloseOnExec(fd)
return fd, err
}
func unmount(dir string, opts *MountOptions) error {
return syscall.Unmount(dir, 0)
}
func getConnection(local *os.File) (int, error) {
var data [4]byte
control := make([]byte, 4*256)
// n, oobn, recvflags, from, errno - todo: error checking.
_, oobn, _, _,
err := syscall.Recvmsg(
int(local.Fd()), data[:], control[:], 0)
if err != nil {
return 0, err
}
message := *(*syscall.Cmsghdr)(unsafe.Pointer(&control[0]))
fd := *(*int32)(unsafe.Pointer(uintptr(unsafe.Pointer(&control[0])) + syscall.SizeofCmsghdr))
if message.Type != syscall.SCM_RIGHTS {
return 0, fmt.Errorf("getConnection: recvmsg returned wrong control type: %d", message.Type)
}
if oobn <= syscall.SizeofCmsghdr {
return 0, fmt.Errorf("getConnection: too short control message. Length: %d", oobn)
}
if fd < 0 {
return 0, fmt.Errorf("getConnection: fd < 0: %d", fd)
}
return int(fd), nil
}
func fusermountBinary() (string, error) {
binPaths := []string{
"/Library/Filesystems/macfuse.fs/Contents/Resources/mount_macfuse",
"/Library/Filesystems/osxfuse.fs/Contents/Resources/mount_osxfuse",
}
for _, path := range binPaths {
if _, err := os.Stat(path); err == nil {
return path, nil
}
}
return "", fmt.Errorf("no FUSE mount utility found")
}
fuse/mount_linux.go
View file @ 43d5f40d
......@@ -68,18 +68,13 @@ func mountDirect(mountPoint string, opts *MountOptions, ready chan<- error) (fd
return
}
// Create a FUSE FS on the specified mount point. The returned
// mount point is always absolute.
func mount(mountPoint string, opts *MountOptions, ready chan<- error) (fd int, err error) {
if opts.DirectMount {
fd, err := mountDirect(mountPoint, opts, ready)
if err == nil {
return fd, nil
} else if opts.Debug {
log.Printf("mount: failed to do direct mount: %s", err)
}
}
// callFusermount calls the `fusermount` suid helper with the right options so
// that it:
// * opens `/dev/fuse`
// * mount()s this file descriptor to `mountPoint`
// * passes this file descriptor back to us via a unix domain socket
// This file descriptor is returned as `fd`.
func callFusermount(mountPoint string, opts *MountOptions) (fd int, err error) {
local, remote, err := unixgramSocketpair()
if err != nil {
return
......@@ -97,6 +92,9 @@ func mount(mountPoint string, opts *MountOptions, ready chan<- error) (fd int, e
if s := opts.optionsStrings(); len(s) > 0 {
cmd = append(cmd, "-o", strings.Join(s, ","))
}
if opts.Debug {
log.Printf("callFusermount: executing %q", cmd)
}
proc, err := os.StartProcess(bin,
cmd,
&os.ProcAttr{
......@@ -121,11 +119,39 @@ func mount(mountPoint string, opts *MountOptions, ready chan<- error) (fd int, e
return -1, err
}
return
}
// Create a FUSE FS on the specified mount point. The returned
// mount point is always absolute.
func mount(mountPoint string, opts *MountOptions, ready chan<- error) (fd int, err error) {
if opts.DirectMount {
fd, err := mountDirect(mountPoint, opts, ready)
if err == nil {
return fd, nil
} else if opts.Debug {
log.Printf("mount: failed to do direct mount: %s", err)
}
}
// Magic `/dev/fd/N` mountpoint. See the docs for NewServer() for how this
// works.
fd = parseFuseFd(mountPoint)
if fd >= 0 {
if opts.Debug {
log.Printf("mount: magic mountpoint %q, using fd %d", mountPoint, fd)
}
} else {
// Usual case: mount via the `fusermount` suid helper
fd, err = callFusermount(mountPoint, opts)
if err != nil {
return
}
}
// golang sets CLOEXEC on file descriptors when they are
// acquired through normal operations (e.g. open).
// Buf for fd, we have to set CLOEXEC manually
syscall.CloseOnExec(fd)
close(ready)
return fd, err
}
......@@ -146,6 +172,9 @@ func unmount(mountPoint string, opts *MountOptions) (err error) {
errBuf := bytes.Buffer{}
cmd := exec.Command(bin, "-u", mountPoint)
cmd.Stderr = &errBuf
if opts.Debug {
log.Printf("unmount: executing %q", cmd.Args)
}
err = cmd.Run()
if errBuf.Len() > 0 {
return fmt.Errorf("%s (code %v)\n",
......@@ -193,7 +222,12 @@ func lookPathFallback(file string, fallbackDir string) (string, error) {
return exec.LookPath(abs)
}
// fusermountBinary returns the path to the `fusermount3` binary, or, if not
// found, the `fusermount` binary.
func fusermountBinary() (string, error) {
if path, err := lookPathFallback("fusermount3", "/bin"); err == nil {
return path, nil
}
return lookPathFallback("fusermount", "/bin")
}
......
fuse/mount_linux_test.go 0 → 100644
View file @ 43d5f40d
package fuse
import (
"fmt"
"io/ioutil"
"syscall"
"testing"
)
// TestMountDevFd tests the special `/dev/fd/N` mountpoint syntax, where a
// privileged parent process opens /dev/fuse and calls mount() for us.
//
// In this test, we simulate a privileged parent by using the `fusermount` suid
// helper.
func TestMountDevFd(t *testing.T) {
realMountPoint, err := ioutil.TempDir("", t.Name())
if err != nil {
t.Fatal(err)
}
defer syscall.Rmdir(realMountPoint)
// Call the fusermount suid helper to obtain the file descriptor in place
// of a privileged parent.
var fuOpts MountOptions
fd, err := callFusermount(realMountPoint, &fuOpts)
if err != nil {
t.Fatal(err)
}
fdMountPoint := fmt.Sprintf("/dev/fd/%d", fd)
// Real test starts here:
// See if we can feed fdMountPoint to NewServer
fs := NewDefaultRawFileSystem()
opts := MountOptions{
Debug: true,
}
srv, err := NewServer(fs, fdMountPoint, &opts)
if err != nil {
t.Fatal(err)
}
go srv.Serve()
if err := srv.WaitMount(); err != nil {
t.Fatal(err)
}
// If we are actually mounted, we should get ENOSYS.
//
// This won't deadlock despite pollHack not working for `/dev/fd/N` mounts
// because functions in the syscall package don't use the poller.
var st syscall.Stat_t
err = syscall.Stat(realMountPoint, &st)
if err != syscall.ENOSYS {
t.Errorf("expected ENOSYS, got %v", err)
}
// Cleanup is somewhat tricky because `srv` does not know about
// `realMountPoint`, so `srv.Unmount()` cannot work.
//
// A normal user has to call `fusermount -u` for themselves to unmount.
// But in this test we can monkey-patch `srv.mountPoint`.
srv.mountPoint = realMountPoint
if err := srv.Unmount(); err != nil {
t.Error(err)
}
}
// TestMountMaxWrite makes sure that mounting works with all MaxWrite settings.
// We used to fail with EINVAL below 8k because readPool got too small.
func TestMountMaxWrite(t *testing.T) {
opts := []MountOptions{
{MaxWrite: 0}, // go-fuse default
{MaxWrite: 1},
{MaxWrite: 123},
{MaxWrite: 1 * 1024},
{MaxWrite: 4 * 1024},
{MaxWrite: 8 * 1024},
{MaxWrite: 64 * 1024}, // go-fuse default
{MaxWrite: 128 * 1024}, // limit in Linux v4.19 and older
{MaxWrite: 999 * 1024},
{MaxWrite: 1024 * 1024}, // limit in Linux v4.20+
}
for _, o := range opts {
name := fmt.Sprintf("MaxWrite%d", o.MaxWrite)
t.Run(name, func(t *testing.T) {
mnt, err := ioutil.TempDir("", name)
if err != nil {
t.Fatal(err)
}
fs := NewDefaultRawFileSystem()
srv, err := NewServer(fs, mnt, &o)
if err != nil {
t.Error(err)
} else {
go srv.Serve()
srv.Unmount()
}
})
}
}
fuse/opcode.go
View file @ 43d5f40d
......@@ -10,6 +10,7 @@ import (
"log"
"reflect"
"runtime"
"syscall"
"time"
"unsafe"
)
......@@ -100,6 +101,18 @@ func doInit(server *Server, req *request) {
server.kernelSettings.Flags |= CAP_FLOCK_LOCKS | CAP_POSIX_LOCKS
}
if server.opts.EnableAcl {
server.kernelSettings.Flags |= CAP_POSIX_ACL
}
if server.opts.SyncRead {
// Clear CAP_ASYNC_READ
server.kernelSettings.Flags &= ^uint32(CAP_ASYNC_READ)
}
if server.opts.DisableReadDirPlus {
// Clear CAP_READDIRPLUS
server.kernelSettings.Flags &= ^uint32(CAP_READDIRPLUS)
}
dataCacheMode := input.Flags & CAP_AUTO_INVAL_DATA
if server.opts.ExplicitDataCacheControl {
// we don't want CAP_AUTO_INVAL_DATA even if we cannot go into fully explicit mode
......@@ -319,7 +332,7 @@ func doBatchForget(server *Server, req *request) {
forgets := *(*[]_ForgetOne)(unsafe.Pointer(h))
for i, f := range forgets {
if server.opts.Debug {
log.Printf("doBatchForget: rx %d %d/%d: FORGET i%d {Nlookup=%d}",
log.Printf("doBatchForget: rx %d %d/%d: FORGET n%d {Nlookup=%d}",
req.inHeader.Unique, i+1, len(forgets), f.NodeId, f.Nlookup)
}
if f.NodeId == pollHackInode {
......@@ -439,7 +452,7 @@ func doStatFs(server *Server, req *request) {
}
func doIoctl(server *Server, req *request) {
req.status = ENOSYS
req.status = Status(syscall.ENOTTY)
}
func doDestroy(server *Server, req *request) {
......@@ -732,6 +745,7 @@ func init() {
_OP_SETATTR: func(ptr unsafe.Pointer) interface{} { return (*AttrOut)(ptr) },
_OP_INIT: func(ptr unsafe.Pointer) interface{} { return (*InitOut)(ptr) },
_OP_MKDIR: func(ptr unsafe.Pointer) interface{} { return (*EntryOut)(ptr) },
_OP_MKNOD: func(ptr unsafe.Pointer) interface{} { return (*EntryOut)(ptr) },
_OP_NOTIFY_INVAL_ENTRY: func(ptr unsafe.Pointer) interface{} { return (*NotifyInvalEntryOut)(ptr) },
_OP_NOTIFY_INVAL_INODE: func(ptr unsafe.Pointer) interface{} { return (*NotifyInvalInodeOut)(ptr) },
_OP_NOTIFY_STORE_CACHE: func(ptr unsafe.Pointer) interface{} { return (*NotifyStoreOut)(ptr) },
......
fuse/poll.go
View file @ 43d5f40d
......@@ -28,15 +28,25 @@ func doPollHackLookup(ms *Server, req *request) {
Fh: pollHackInode,
}
req.status = OK
case _OP_GETATTR:
case _OP_GETATTR, _OP_SETATTR:
out := (*AttrOut)(req.outData())
out.Attr = attr
req.status = OK
case _OP_GETXATTR:
// Kernel will try to read acl xattrs. Pretend we don't have any.
req.status = ENODATA
case _OP_POLL:
req.status = ENOSYS
case _OP_ACCESS, _OP_FLUSH, _OP_RELEASE:
// Avoid upsetting the OSX mount process.
req.status = OK
default:
// We want to avoid switching off features through our
// poll hack, so don't use ENOSYS
// poll hack, so don't use ENOSYS. It would be nice if
// we could transmit no error code at all, but for
// some opcodes, we'd have to invent credible data to
// return as well.
req.status = ERANGE
}
}
fuse/poll_darwin.go
View file @ 43d5f40d
......@@ -32,7 +32,7 @@ func pollHack(mountPoint string) error {
POLLHUP = 0x10
)
fd, err := syscall.Open(filepath.Join(mountPoint, pollHackName), syscall.O_CREAT|syscall.O_TRUNC|syscall.O_RDWR, 0644)
fd, err := syscall.Open(filepath.Join(mountPoint, pollHackName), syscall.O_RDONLY, 0)
if err != nil {
return err
}
......
fuse/print.go
View file @ 43d5f40d
......@@ -166,17 +166,17 @@ func (in *OpenOut) string() string {
}
func (in *InitIn) string() string {
return fmt.Sprintf("{%d.%d Ra 0x%x %s}",
return fmt.Sprintf("{%d.%d Ra %d %s}",
in.Major, in.Minor, in.MaxReadAhead,
flagString(initFlagNames, int64(in.Flags), ""))
}
func (o *InitOut) string() string {
return fmt.Sprintf("{%d.%d Ra 0x%x %s %d/%d Wr 0x%x Tg 0x%x}",
return fmt.Sprintf("{%d.%d Ra %d %s %d/%d Wr %d Tg %d MaxPages %d}",
o.Major, o.Minor, o.MaxReadAhead,
flagString(initFlagNames, int64(o.Flags), ""),
o.CongestionThreshold, o.MaxBackground, o.MaxWrite,
o.TimeGran)
o.TimeGran, o.MaxPages)
}
func (s *FsyncIn) string() string {
......@@ -219,13 +219,13 @@ func ft(tsec uint64, tnsec uint32) float64 {
// Returned by LOOKUP
func (o *EntryOut) string() string {
return fmt.Sprintf("{i%d g%d tE=%gs tA=%gs %v}",
return fmt.Sprintf("{n%d g%d tE=%gs tA=%gs %v}",
o.NodeId, o.Generation, ft(o.EntryValid, o.EntryValidNsec),
ft(o.AttrValid, o.AttrValidNsec), &o.Attr)
}
func (o *CreateOut) string() string {
return fmt.Sprintf("{i%d g%d %v %v}", o.NodeId, o.Generation, &o.EntryOut, &o.OpenOut)
return fmt.Sprintf("{n%d g%d %v %v}", o.NodeId, o.Generation, &o.EntryOut, &o.OpenOut)
}
func (o *StatfsOut) string() string {
......@@ -248,11 +248,11 @@ func (o *NotifyInvalDeleteOut) string() string {
}
func (o *NotifyStoreOut) string() string {
return fmt.Sprintf("{i%d [%d +%d)}", o.Nodeid, o.Offset, o.Size)
return fmt.Sprintf("{n%d [%d +%d)}", o.Nodeid, o.Offset, o.Size)
}
func (o *NotifyRetrieveOut) string() string {
return fmt.Sprintf("{> %d: i%d [%d +%d)}", o.NotifyUnique, o.Nodeid, o.Offset, o.Size)
return fmt.Sprintf("{> %d: n%d [%d +%d)}", o.NotifyUnique, o.Nodeid, o.Offset, o.Size)
}
func (i *NotifyRetrieveIn) string() string {
......@@ -265,7 +265,7 @@ func (f *FallocateIn) string() string {
}
func (f *LinkIn) string() string {
return fmt.Sprintf("{Oldnodeid: %d}", f.Oldnodeid)
return fmt.Sprintf("{Oldnodeid: n%d}", f.Oldnodeid)
}
func (o *WriteOut) string() string {
......@@ -273,7 +273,7 @@ func (o *WriteOut) string() string {
}
func (i *CopyFileRangeIn) string() string {
return fmt.Sprintf("{Fh %d [%d +%d) => i%d Fh %d [%d, %d)}",
return fmt.Sprintf("{Fh %d [%d +%d) => n%d Fh %d [%d, %d)}",
i.FhIn, i.OffIn, i.Len, i.NodeIdOut, i.FhOut, i.OffOut, i.Len)
}
......
fuse/print_darwin.go
View file @ 43d5f40d
......@@ -18,13 +18,13 @@ func (a *Attr) string() string {
return fmt.Sprintf(
"{M0%o SZ=%d L=%d "+
"%d:%d "+
"%d %d:%d "+
"B%d*%d i%d:%d "+
"A %f "+
"M %f "+
"C %f}",
a.Mode, a.Size, a.Nlink,
a.Uid, a.Gid,
a.Blocks,
a.Blocks, a.Blksize,
a.Rdev, a.Ino, ft(a.Atime, a.Atimensec), ft(a.Mtime, a.Mtimensec),
ft(a.Ctime, a.Ctimensec))
}
......
fuse/request.go
View file @ 43d5f40d
......@@ -105,7 +105,7 @@ func (r *request) InputDebug() string {
names += fmt.Sprintf("%s %db", data, len(r.arg))
}
return fmt.Sprintf("rx %d: %s i%d %s%s",
return fmt.Sprintf("rx %d: %s n%d %s%s",
r.inHeader.Unique, operationName(r.inHeader.Opcode), r.inHeader.NodeId,
val, names)
}
......
fuse/request_darwin.go
View file @ 43d5f40d
......@@ -8,6 +8,6 @@ const outputHeaderSize = 200
const (
_FUSE_KERNEL_VERSION = 7
_MINIMUM_MINOR_VERSION = 8
_OUR_MINOR_VERSION = 8
_MINIMUM_MINOR_VERSION = 12
_OUR_MINOR_VERSION = 12
)
fuse/server.go
View file @ 43d5f40d
......@@ -9,8 +9,10 @@ import (
"log"
"math"
"os"
"path"
"path/filepath"
"runtime"
"strconv"
"strings"
"sync"
"syscall"
......@@ -21,6 +23,13 @@ import (
const (
// The kernel caps writes at 128k.
MAX_KERNEL_WRITE = 128 * 1024
// Linux kernel constant from include/uapi/linux/fuse.h
// Reads from /dev/fuse that are smaller fail with EINVAL.
_FUSE_MIN_READ_BUFFER = 8192
minMaxReaders = 2
maxMaxReaders = 16
)
// Server contains the logic for reading from the FUSE device and
......@@ -40,6 +49,9 @@ type Server struct {
opts *MountOptions
// maxReaders is the maximum number of goroutines reading requests
maxReaders int
// Pools for []byte
buffers bufferPool
......@@ -102,10 +114,20 @@ func (ms *Server) RecordLatencies(l LatencyMap) {
// Unmount calls fusermount -u on the mount. This has the effect of
// shutting down the filesystem. After the Server is unmounted, it
// should be discarded.
//
// Does not work when we were mounted with the magic /dev/fd/N mountpoint syntax,
// as we do not know the real mountpoint. Unmount using
//
// fusermount -u /path/to/real/mountpoint
//
/// in this case.
func (ms *Server) Unmount() (err error) {
if ms.mountPoint == "" {
return nil
}
if parseFuseFd(ms.mountPoint) >= 0 {
return fmt.Errorf("Cannot unmount magic mountpoint %q. Please use `fusermount -u REALMOUNTPOINT` instead.", ms.mountPoint)
}
delay := time.Duration(0)
for try := 0; try < 5; try++ {
err = unmount(ms.mountPoint, ms.opts)
......@@ -128,7 +150,11 @@ func (ms *Server) Unmount() (err error) {
return err
}
// NewServer creates a server and attaches it to the given directory.
// NewServer creates a FUSE server and attaches ("mounts") it to the
// `mountPoint` directory.
//
// See the "Mount styles" section in the package documentation if you want to
// know about the inner workings of the mount process. Usually you do not.
func NewServer(fs RawFileSystem, mountPoint string, opts *MountOptions) (*Server, error) {
if opts == nil {
opts = &MountOptions{
......@@ -161,9 +187,17 @@ func NewServer(fs RawFileSystem, mountPoint string, opts *MountOptions) (*Server
}
}
maxReaders := runtime.GOMAXPROCS(0)
if maxReaders < minMaxReaders {
maxReaders = minMaxReaders
} else if maxReaders > maxMaxReaders {
maxReaders = maxMaxReaders
}
ms := &Server{
fileSystem: fs,
opts: &o,
maxReaders: maxReaders,
retrieveTab: make(map[uint64]*retrieveCacheRequest),
// OSX has races when multiple routines read from the
// FUSE device: on unmount, sometime some reads do not
......@@ -177,8 +211,12 @@ func NewServer(fs RawFileSystem, mountPoint string, opts *MountOptions) (*Server
}
}
ms.readPool.New = func() interface{} {
buf := make([]byte, o.MaxWrite+int(maxInputSize)+logicalBlockSize)
buf = alignSlice(buf, unsafe.Sizeof(WriteIn{}), logicalBlockSize, uintptr(o.MaxWrite)+maxInputSize)
targetSize := o.MaxWrite + int(maxInputSize)
if targetSize < _FUSE_MIN_READ_BUFFER {
targetSize = _FUSE_MIN_READ_BUFFER
}
buf := make([]byte, targetSize+logicalBlockSize)
buf = alignSlice(buf, unsafe.Sizeof(WriteIn{}), logicalBlockSize, uintptr(targetSize))
return buf
}
mountPoint = filepath.Clean(mountPoint)
......@@ -224,6 +262,13 @@ func (o *MountOptions) optionsStrings() []string {
r = append(r, "subtype="+o.Name)
}
// OSXFUSE applies a 60-second timeout for file operations. This
// is inconsistent with how FUSE works on Linux, where operations
// last as long as the daemon is willing to let them run.
if runtime.GOOS == "darwin" {
r = append(r, "daemon_timeout=0")
}
return r
}
......@@ -238,10 +283,6 @@ func (ms *Server) DebugData() string {
return fmt.Sprintf("readers: %d", r)
}
// What is a good number? Maybe the number of CPUs?
// XXX -> use fuse fd cloning
const _MAX_READERS = 4 // <-- XXX NOTE
// handleEINTR retries the given function until it doesn't return syscall.EINTR.
// This is similar to the HANDLE_EINTR() macro from Chromium ( see
// https://code.google.com/p/chromium/codesearch#chromium/src/base/posix/eintr_wrapper.h
......@@ -264,17 +305,17 @@ func handleEINTR(fn func() error) (err error) {
// Returns a new request, or error. In case exitIdle is given, returns
// nil, OK if we have too many readers already.
func (ms *Server) readRequest(exitIdle bool) (req *request, code Status) {
req = ms.reqPool.Get().(*request)
dest := ms.readPool.Get().([]byte)
ms.reqMu.Lock()
if ms.reqReaders > _MAX_READERS {
if ms.reqReaders > ms.maxReaders {
ms.reqMu.Unlock()
return nil, OK
}
ms.reqReaders++
ms.reqMu.Unlock()
req = ms.reqPool.Get().(*request)
dest := ms.readPool.Get().([]byte)
var n int
err := handleEINTR(func() error {
var err error
......@@ -475,8 +516,15 @@ func (ms *Server) handleRequest(req *request) Status {
errNo := ms.write(req)
if errNo != 0 {
log.Printf("writer: Write/Writev failed, err: %v. opcode: %v",
errNo, operationName(req.inHeader.Opcode))
// Unless debugging is enabled, ignore ENOENT for INTERRUPT responses
// which indicates that the referred request is no longer known by the
// kernel. This is a normal if the referred request already has
// completed.
if ms.opts.Debug || !(req.inHeader.Opcode == _OP_INTERRUPT && errNo == ENOENT) {
log.Printf("writer: Write/Writev failed, err: %v. opcode: %v",
errNo, operationName(req.inHeader.Opcode))
}
}
ms.returnRequest(req)
return Status(errNo)
......@@ -859,5 +907,24 @@ func (ms *Server) WaitMount() error {
if err != nil {
return err
}
if parseFuseFd(ms.mountPoint) >= 0 {
// Magic `/dev/fd/N` mountpoint. We don't know the real mountpoint, so
// we cannot run the poll hack.
return nil
}
return pollHack(ms.mountPoint)
}
// parseFuseFd checks if `mountPoint` is the special form /dev/fd/N (with N >= 0),
// and returns N in this case. Returns -1 otherwise.
func parseFuseFd(mountPoint string) (fd int) {
dir, file := path.Split(mountPoint)
if dir != "/dev/fd/" {
return -1
}
fd, err := strconv.Atoi(file)
if err != nil || fd <= 0 {
return -1
}
return fd
}
fuse/test/cachecontrol_test.go
View file @ 43d5f40d
......@@ -189,11 +189,19 @@ func TestCacheControl(t *testing.T) {
defer func() {
xmunmap(fmmap)
}()
xmlock(fmmap)
// assertMmapRead asserts that file's mmaped memory reads as dataOK.
assertMmapRead := func(subj, dataOK string) {
t.Helper()
// Use the Mlock() syscall to get the mmap'ed range into the kernel
// cache again, triggering FUSE reads as neccessary. A blocked syscall does
// not count towards GOMAXPROCS, so there should be a thread available
// to handle the FUSE reads.
// If we don't Mlock() first, the memory comparison triggers a page
// fault, which blocks the thread, and deadlocks the test reliably at
// GOMAXPROCS=1.
// Fixes https://github.com/hanwen/go-fuse/issues/261 .
xmlock(fmmap)
if string(fmmap) != dataOK {
t.Fatalf("%s: file mmap: got %q ; want %q", subj, fmmap, dataOK)
}
......
fuse/test/delete_linux_test.go
View file @ 43d5f40d
......@@ -81,8 +81,7 @@ func TestDeleteNotify(t *testing.T) {
return
}
buf := bytes.Buffer{}
cmd := exec.Command("/usr/bin/tail", "-f", "testfile")
cmd.Dir = mnt + "/testdir"
cmd := exec.Command("/usr/bin/tail", "-f", mnt+"/testdir/testfile")
cmd.Stdin = &buf
cmd.Stdout = &bytes.Buffer{}
cmd.Stderr = os.Stderr
......
fuse/test/loopback_linux_test.go
View file @ 43d5f40d
......@@ -5,8 +5,11 @@
package test
import (
"flag"
"fmt"
"io/ioutil"
"os"
"path/filepath"
"syscall"
"testing"
"time"
......@@ -14,8 +17,18 @@ import (
"golang.org/x/sys/unix"
"github.com/hanwen/go-fuse/v2/fuse"
"github.com/hanwen/go-fuse/v2/internal/testutil"
)
var enableOverlayfsTest bool
var enableOverlayfsTestFlag string = "test.overlayfs"
func init() {
// flag to enable test with overayfs. This would not work on kernel 5.15
// so don't enable on that kernel.
flag.BoolVar(&enableOverlayfsTest, enableOverlayfsTestFlag, false, "enable tests with overlayfs (would fail on kernel 5.15)")
}
func TestTouch(t *testing.T) {
ts := NewTestCase(t)
defer ts.Cleanup()
......@@ -118,6 +131,40 @@ func clearStatfs(s *syscall.Statfs_t) {
s.Flags = 0
}
// Check that fuse mount can serve as a overlayfs lowerdir.
func TestOverlayfs(t *testing.T) {
if !enableOverlayfsTest {
t.Skipf("this test must be enabled through the flag %q", enableOverlayfsTestFlag)
}
if os.Getuid() != 0 {
t.Skip("this test requires root")
}
tc := NewTestCase(t)
defer tc.Cleanup()
testfile := "test"
content := randomData(125)
tc.Mkdir(tc.origSubdir, 0777)
tc.WriteFile(filepath.Join(tc.origSubdir, testfile), content, 0700)
tmpMergedDir := testutil.TempDir()
defer os.RemoveAll(tmpMergedDir)
tmpWorkDir := testutil.TempDir()
defer os.RemoveAll(tmpWorkDir)
tmpUpperDir := testutil.TempDir()
defer os.RemoveAll(tmpUpperDir)
if err := unix.Mount("overlay", tmpMergedDir, "overlay", 0,
fmt.Sprintf("lowerdir=%s,upperdir=%s,workdir=%s", tc.mnt, tmpUpperDir, tmpWorkDir)); err != nil {
t.Fatalf("failed to mount overlay: %v", err)
}
defer unix.Unmount(tmpMergedDir, 0)
err := os.Chtimes(filepath.Join(tmpMergedDir, "subdir", testfile), time.Unix(42, 0), time.Unix(43, 0))
if err != nil {
t.Fatalf("Chtimes failed: %v", err)
}
}
func TestFallocate(t *testing.T) {
ts := NewTestCase(t)
defer ts.Cleanup()
......
fuse/test/node_parallel_lookup_test.go
View file @ 43d5f40d
......@@ -98,6 +98,11 @@ func TestNodeParallelLookup(t *testing.T) {
}
}()
// the test will deadlock if the client cannot issue several lookups simultaneously
if srv.KernelSettings().Flags & fuse.CAP_PARALLEL_DIROPS == 0 {
t.Skip("Kernel serializes dir lookups")
}
// spawn 2 threads to access the files in parallel
// this will deadlock if nodefs does not allow simultaneous Lookups to be handled.
// see https://github.com/hanwen/go-fuse/commit/d0fca860 for context.
......
fuse/types.go
View file @ 43d5f40d
......@@ -265,6 +265,13 @@ type OpenOut struct {
}
// To be set in InitIn/InitOut.Flags.
//
// Keep in sync with either of
// * https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/fuse.h
// * https://github.com/libfuse/libfuse/blob/master/include/fuse_kernel.h
// but NOT with
// * https://github.com/libfuse/libfuse/blob/master/include/fuse_common.h
// This file has CAP_HANDLE_KILLPRIV and CAP_POSIX_ACL reversed!
const (
CAP_ASYNC_READ = (1 << 0)
CAP_POSIX_LOCKS = (1 << 1)
......
fuse/types_darwin.go
View file @ 43d5f40d
......@@ -30,8 +30,10 @@ type Attr struct {
Mode uint32
Nlink uint32
Owner
Rdev uint32
Flags_ uint32 // OS X
Rdev uint32
Flags_ uint32 // OS X
Blksize uint32
Padding uint32
}
const (
......@@ -67,38 +69,52 @@ const (
type GetAttrIn struct {
InHeader
Flags_ uint32
Dummy uint32
Fh_ uint64
}
func (g *GetAttrIn) Flags() uint32 {
return 0
return g.Flags_
}
func (g *GetAttrIn) Fh() uint64 {
return 0
return g.Fh_
}
// Uses OpenIn struct for create.
type CreateIn struct {
InHeader
Flags uint32
Mode uint32
// Mode for the new file; already takes Umask into account.
Mode uint32
// Umask used for this create call.
Umask uint32
Padding uint32
}
type MknodIn struct {
InHeader
Mode uint32
Rdev uint32
// Mode to use, including the Umask value
Mode uint32
Rdev uint32
Umask uint32
Padding uint32
}
type ReadIn struct {
InHeader
Fh uint64
Offset uint64
Size uint32
ReadFlags uint32
LockOwner uint64
Flags uint32
Padding uint32
}
type WriteIn struct {
......@@ -107,6 +123,9 @@ type WriteIn struct {
Offset uint64
Size uint32
WriteFlags uint32
LockOwner uint64
Flags uint32
Padding uint32
}
type SetXAttrIn struct {
......
go.mod
View file @ 43d5f40d
module github.com/hanwen/go-fuse/v2
require (
github.com/hanwen/go-fuse v1.0.0
github.com/kylelemons/godebug v0.0.0-20170820004349-d65d576e9348
golang.org/x/sync v0.0.0-20201207232520-09787c993a3a
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522
)
......
go.sum
View file @ 43d5f40d
github.com/hanwen/go-fuse v1.0.0 h1:GxS9Zrn6c35/BnfiVsZVWmsG803xwE7eVRDvcf/BEVc=
github.com/hanwen/go-fuse v1.0.0/go.mod h1:unqXarDXqzAk0rt98O2tVndEPIpUgLD9+rwFisZH3Ok=
github.com/kylelemons/godebug v0.0.0-20170820004349-d65d576e9348 h1:MtvEpTB6LX3vkb4ax0b5D2DHbNAUsen0Gx5wZoq3lV4=
github.com/kylelemons/godebug v0.0.0-20170820004349-d65d576e9348/go.mod h1:B69LEHPfb2qLo0BaaOLcbitczOKLWTsrBG9LczfCD4k=
golang.org/x/sync v0.0.0-20201207232520-09787c993a3a h1:DcqTD9SDLc+1P/r1EmRBwnVsrOwW+kk2vWf9n+1sGhs=
golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522 h1:Ve1ORMCxvRmSXBwJK+t3Oy+V2vRW2OetUQBq4rJIkZE=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
posixtest/test.go
View file @ 43d5f40d
......@@ -487,11 +487,16 @@ func ReadDir(t *testing.T, mnt string) {
got[e] = true
}
if len(got) != len(want) {
t.Errorf("got %d entries, want %d", len(got), len(want))
t.Errorf("mismatch got %d want %d", len(got), len(want))
}
for k := range got {
if !want[k] {
t.Errorf("got unknown name %q", k)
t.Errorf("got extra entry %q", k)
}
}
for k := range want {
if !got[k] {
t.Errorf("missing entry %q", k)
}
}
}
......
unionfs/autounion.go deleted 100644 → 0
View file @ 51ae026e
// Copyright 2016 the Go-FUSE Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package unionfs
import (
"fmt"
"io/ioutil"
"log"
"os"
"path/filepath"
"strings"
"sync"
"syscall"
"time"
"github.com/hanwen/go-fuse/v2/fuse"
"github.com/hanwen/go-fuse/v2/fuse/nodefs"
"github.com/hanwen/go-fuse/v2/fuse/pathfs"
)
type knownFs struct {
unionFS pathfs.FileSystem
nodeFS *pathfs.PathNodeFs
}
// Creates unions for all files under a given directory,
// walking the tree and looking for directories D which have a
// D/READONLY symlink.
//
// A union for A/B/C will placed under directory A-B-C.
type autoUnionFs struct {
pathfs.FileSystem
debug bool
lock sync.RWMutex
zombies map[string]bool
knownFileSystems map[string]knownFs
nameRootMap map[string]string
root string
nodeFs *pathfs.PathNodeFs
options *AutoUnionFsOptions
}
type AutoUnionFsOptions struct {
UnionFsOptions
nodefs.Options
pathfs.PathNodeFsOptions
// If set, run updateKnownFses() after mounting.
UpdateOnMount bool
// If set hides the _READONLY file.
HideReadonly bool
// Expose this version in /status/gounionfs_version
Version string
}
const (
_READONLY = "READONLY"
_STATUS = "status"
_CONFIG = "config"
_DEBUG = "debug"
_ROOT = "root"
_VERSION = "gounionfs_version"
_SCAN_CONFIG = ".scan_config"
)
func NewAutoUnionFs(directory string, options AutoUnionFsOptions) pathfs.FileSystem {
if options.HideReadonly {
options.HiddenFiles = append(options.HiddenFiles, _READONLY)
}
a := &autoUnionFs{
knownFileSystems: make(map[string]knownFs),
nameRootMap: make(map[string]string),
zombies: make(map[string]bool),
options: &options,
FileSystem: pathfs.NewDefaultFileSystem(),
}
directory, err := filepath.Abs(directory)
if err != nil {
panic("filepath.Abs returned err")
}
a.root = directory
return a
}
func (fs *autoUnionFs) String() string {
return fmt.Sprintf("autoUnionFs(%s)", fs.root)
}
func (fs *autoUnionFs) OnMount(nodeFs *pathfs.PathNodeFs) {
fs.nodeFs = nodeFs
if fs.options.UpdateOnMount {
time.AfterFunc(100*time.Millisecond, func() { fs.updateKnownFses() })
}
}
func (fs *autoUnionFs) addAutomaticFs(roots []string) {
relative := strings.TrimLeft(strings.Replace(roots[0], fs.root, "", -1), "/")
name := strings.Replace(relative, "/", "-", -1)
if fs.getUnionFs(name) == nil {
fs.addFs(name, roots)
}
}
func (fs *autoUnionFs) createFs(name string, roots []string) fuse.Status {
fs.lock.Lock()
defer fs.lock.Unlock()
if fs.zombies[name] {
log.Printf("filesystem named %q is being removed", name)
return fuse.EBUSY
}
for workspace, root := range fs.nameRootMap {
if root == roots[0] && workspace != name {
log.Printf("Already have a union FS for directory %s in workspace %s",
roots[0], workspace)
return fuse.EBUSY
}
}
known := fs.knownFileSystems[name]
if known.unionFS != nil {
log.Println("Already have a workspace:", name)
return fuse.EBUSY
}
ufs, err := NewUnionFsFromRoots(roots, &fs.options.UnionFsOptions, true)
if err != nil {
log.Println("Could not create UnionFs:", err)
return fuse.EPERM
}
log.Printf("Adding workspace %v for roots %v", name, ufs.String())
nfs := pathfs.NewPathNodeFs(ufs, &fs.options.PathNodeFsOptions)
code := fs.nodeFs.Mount(name, nfs.Root(), &fs.options.Options)
if code.Ok() {
fs.knownFileSystems[name] = knownFs{
ufs,
nfs,
}
fs.nameRootMap[name] = roots[0]
}
return code
}
func (fs *autoUnionFs) rmFs(name string) (code fuse.Status) {
fs.lock.Lock()
defer fs.lock.Unlock()
if fs.zombies[name] {
return fuse.ENOENT
}
known := fs.knownFileSystems[name]
if known.unionFS == nil {
return fuse.ENOENT
}
root := fs.nameRootMap[name]
delete(fs.knownFileSystems, name)
delete(fs.nameRootMap, name)
fs.zombies[name] = true
fs.lock.Unlock()
code = fs.nodeFs.Unmount(name)
fs.lock.Lock()
delete(fs.zombies, name)
if !code.Ok() {
// Reinstate.
log.Printf("Unmount failed for %s. Code %v", name, code)
fs.knownFileSystems[name] = known
fs.nameRootMap[name] = root
}
return code
}
func (fs *autoUnionFs) addFs(name string, roots []string) (code fuse.Status) {
if name == _CONFIG || name == _STATUS || name == _SCAN_CONFIG {
return fuse.EINVAL
}
return fs.createFs(name, roots)
}
func (fs *autoUnionFs) getRoots(path string) []string {
ro := filepath.Join(path, _READONLY)
fi, err := os.Lstat(ro)
fiDir, errDir := os.Stat(ro)
if err != nil || errDir != nil {
return nil
}
if fi.Mode()&os.ModeSymlink != 0 && fiDir.IsDir() {
// TODO - should recurse and chain all READONLYs
// together.
return []string{path, ro}
}
return nil
}
func (fs *autoUnionFs) visit(path string, fi os.FileInfo, err error) error {
if fi != nil && fi.IsDir() {
roots := fs.getRoots(path)
if roots != nil {
fs.addAutomaticFs(roots)
}
}
return nil
}
func (fs *autoUnionFs) updateKnownFses() {
// We unroll the first level of entries in the root manually in order
// to allow symbolic links on that level.
directoryEntries, err := ioutil.ReadDir(fs.root)
if err == nil {
for _, dir := range directoryEntries {
if dir.IsDir() || dir.Mode()&os.ModeSymlink != 0 {
path := filepath.Join(fs.root, dir.Name())
dir, _ = os.Stat(path)
fs.visit(path, dir, nil)
filepath.Walk(path,
func(path string, fi os.FileInfo, err error) error {
return fs.visit(path, fi, err)
})
}
}
}
}
func (fs *autoUnionFs) Readlink(path string, context *fuse.Context) (out string, code fuse.Status) {
comps := strings.Split(path, string(filepath.Separator))
if comps[0] == _STATUS && comps[1] == _ROOT {
return fs.root, fuse.OK
}
if comps[0] != _CONFIG {
return "", fuse.ENOENT
}
name := comps[1]
fs.lock.RLock()
defer fs.lock.RUnlock()
root, ok := fs.nameRootMap[name]
if ok {
return root, fuse.OK
}
return "", fuse.ENOENT
}
func (fs *autoUnionFs) getUnionFs(name string) pathfs.FileSystem {
fs.lock.RLock()
defer fs.lock.RUnlock()
return fs.knownFileSystems[name].unionFS
}
func (fs *autoUnionFs) Symlink(pointedTo string, linkName string, context *fuse.Context) (code fuse.Status) {
comps := strings.Split(linkName, "/")
if len(comps) != 2 {
return fuse.EPERM
}
if comps[0] == _CONFIG {
roots := fs.getRoots(pointedTo)
if roots == nil {
return fuse.Status(syscall.ENOTDIR)
}
name := comps[1]
return fs.addFs(name, roots)
}
return fuse.EPERM
}
func (fs *autoUnionFs) Unlink(path string, context *fuse.Context) (code fuse.Status) {
comps := strings.Split(path, "/")
if len(comps) != 2 {
return fuse.EPERM
}
if comps[0] == _CONFIG && comps[1] != _SCAN_CONFIG {
code = fs.rmFs(comps[1])
} else {
code = fuse.ENOENT
}
return code
}
// Must define this, because ENOSYS will suspend all GetXAttr calls.
func (fs *autoUnionFs) GetXAttr(name string, attr string, context *fuse.Context) ([]byte, fuse.Status) {
return nil, fuse.ENOATTR
}
func (fs *autoUnionFs) GetAttr(path string, context *fuse.Context) (*fuse.Attr, fuse.Status) {
a := &fuse.Attr{
Owner: *fuse.CurrentOwner(),
}
if path == "" || path == _CONFIG || path == _STATUS {
a.Mode = fuse.S_IFDIR | 0755
return a, fuse.OK
}
if path == filepath.Join(_STATUS, _VERSION) {
a.Mode = fuse.S_IFREG | 0644
a.Size = uint64(len(fs.options.Version))
return a, fuse.OK
}
if path == filepath.Join(_STATUS, _DEBUG) {
a.Mode = fuse.S_IFREG | 0644
a.Size = uint64(len(fs.DebugData()))
return a, fuse.OK
}
if path == filepath.Join(_STATUS, _ROOT) {
a.Mode = syscall.S_IFLNK | 0644
return a, fuse.OK
}
if path == filepath.Join(_CONFIG, _SCAN_CONFIG) {
a.Mode = fuse.S_IFREG | 0644
return a, fuse.OK
}
comps := strings.Split(path, string(filepath.Separator))
if len(comps) > 1 && comps[0] == _CONFIG {
fs := fs.getUnionFs(comps[1])
if fs == nil {
return nil, fuse.ENOENT
}
a.Mode = syscall.S_IFLNK | 0644
return a, fuse.OK
}
return nil, fuse.ENOENT
}
func (fs *autoUnionFs) StatusDir() (stream []fuse.DirEntry, status fuse.Status) {
stream = make([]fuse.DirEntry, 0, 10)
stream = []fuse.DirEntry{
{Name: _VERSION, Mode: fuse.S_IFREG | 0644},
{Name: _DEBUG, Mode: fuse.S_IFREG | 0644},
{Name: _ROOT, Mode: syscall.S_IFLNK | 0644},
}
return stream, fuse.OK
}
func (fs *autoUnionFs) DebugData() string {
conn := fs.nodeFs.Connector()
if conn.Server() == nil {
return "autoUnionFs.mountState not set"
}
setting := conn.Server().KernelSettings()
msg := fmt.Sprintf(
"Version: %v\n"+
"Bufferpool: %v\n"+
"Kernel: %v\n",
fs.options.Version,
conn.Server().DebugData(),
&setting)
if conn != nil {
msg += fmt.Sprintf("Live inodes: %d\n", conn.InodeHandleCount())
}
return msg
}
func (fs *autoUnionFs) Open(path string, flags uint32, context *fuse.Context) (nodefs.File, fuse.Status) {
if path == filepath.Join(_STATUS, _DEBUG) {
if flags&fuse.O_ANYWRITE != 0 {
return nil, fuse.EPERM
}
return nodefs.NewDataFile([]byte(fs.DebugData())), fuse.OK
}
if path == filepath.Join(_STATUS, _VERSION) {
if flags&fuse.O_ANYWRITE != 0 {
return nil, fuse.EPERM
}
return nodefs.NewDataFile([]byte(fs.options.Version)), fuse.OK
}
if path == filepath.Join(_CONFIG, _SCAN_CONFIG) {
if flags&fuse.O_ANYWRITE != 0 {
fs.updateKnownFses()
}
return nodefs.NewDevNullFile(), fuse.OK
}
return nil, fuse.ENOENT
}
func (fs *autoUnionFs) Truncate(name string, offset uint64, context *fuse.Context) (code fuse.Status) {
if name != filepath.Join(_CONFIG, _SCAN_CONFIG) {
log.Println("Huh? Truncating unsupported write file", name)
return fuse.EPERM
}
return fuse.OK
}
func (fs *autoUnionFs) OpenDir(name string, context *fuse.Context) (stream []fuse.DirEntry, status fuse.Status) {
switch name {
case _STATUS:
return fs.StatusDir()
case _CONFIG:
case "/":
name = ""
case "":
default:
log.Printf("Argh! Don't know how to list dir %v", name)
return nil, fuse.ENOSYS
}
fs.lock.RLock()
defer fs.lock.RUnlock()
stream = make([]fuse.DirEntry, 0, len(fs.knownFileSystems)+5)
if name == _CONFIG {
for k := range fs.knownFileSystems {
stream = append(stream, fuse.DirEntry{
Name: k,
Mode: syscall.S_IFLNK | 0644,
})
}
}
if name == "" {
stream = append(stream, fuse.DirEntry{
Name: _CONFIG,
Mode: uint32(fuse.S_IFDIR | 0755),
},
fuse.DirEntry{
Name: _STATUS,
Mode: uint32(fuse.S_IFDIR | 0755),
})
}
return stream, status
}
func (fs *autoUnionFs) StatFs(name string) *fuse.StatfsOut {
return &fuse.StatfsOut{}
}
unionfs/autounion_test.go deleted 100644 → 0
View file @ 51ae026e
// Copyright 2016 the Go-FUSE Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package unionfs
import (
"io/ioutil"
"os"
"testing"
"time"
"github.com/hanwen/go-fuse/v2/fuse"
"github.com/hanwen/go-fuse/v2/fuse/nodefs"
"github.com/hanwen/go-fuse/v2/fuse/pathfs"
"github.com/hanwen/go-fuse/v2/internal/testutil"
)
const entryTTL = 100 * time.Millisecond
var testAOpts = AutoUnionFsOptions{
UnionFsOptions: testOpts,
Options: nodefs.Options{
EntryTimeout: entryTTL,
AttrTimeout: entryTTL,
NegativeTimeout: 0,
Debug: testutil.VerboseTest(),
LookupKnownChildren: true,
},
HideReadonly: true,
Version: "version",
}
func init() {
testAOpts.Options.Debug = testutil.VerboseTest()
}
func WriteFile(t *testing.T, name string, contents string) {
err := ioutil.WriteFile(name, []byte(contents), 0644)
if err != nil {
t.Fatalf("WriteFile failed: %v", err)
}
}
func setup(t *testing.T) (workdir string, server *fuse.Server, cleanup func()) {
wd := testutil.TempDir()
err := os.Mkdir(wd+"/mnt", 0700)
if err != nil {
t.Fatalf("Mkdir failed: %v", err)
}
err = os.Mkdir(wd+"/store", 0700)
if err != nil {
t.Fatalf("Mkdir failed: %v", err)
}
os.Mkdir(wd+"/ro", 0700)
if err != nil {
t.Fatalf("Mkdir failed: %v", err)
}
WriteFile(t, wd+"/ro/file1", "file1")
WriteFile(t, wd+"/ro/file2", "file2")
fs := NewAutoUnionFs(wd+"/store", testAOpts)
nfs := pathfs.NewPathNodeFs(fs, nil)
state, _, err := nodefs.MountRoot(wd+"/mnt", nfs.Root(), &testAOpts.Options)
if err != nil {
t.Fatalf("MountNodeFileSystem failed: %v", err)
}
go state.Serve()
state.WaitMount()
return wd, state, func() {
state.Unmount()
os.RemoveAll(wd)
}
}
func TestDebug(t *testing.T) {
wd, _, clean := setup(t)
defer clean()
c, err := ioutil.ReadFile(wd + "/mnt/status/debug")
if err != nil {
t.Fatalf("ReadFile failed: %v", err)
}
if len(c) == 0 {
t.Fatal("No debug found.")
}
}
func TestVersion(t *testing.T) {
wd, _, clean := setup(t)
defer clean()
c, err := ioutil.ReadFile(wd + "/mnt/status/gounionfs_version")
if err != nil {
t.Fatalf("ReadFile failed: %v", err)
}
if len(c) == 0 {
t.Fatal("No version found.")
}
}
func TestAutoFsSymlink(t *testing.T) {
wd, server, clean := setup(t)
defer clean()
err := os.Mkdir(wd+"/store/backing1", 0755)
if err != nil {
t.Fatalf("Mkdir failed: %v", err)
}
err = os.Symlink(wd+"/ro", wd+"/store/backing1/READONLY")
if err != nil {
t.Fatalf("Symlink failed: %v", err)
}
err = os.Symlink(wd+"/store/backing1", wd+"/mnt/config/manual1")
if err != nil {
t.Fatalf("Symlink failed: %v", err)
}
fi, err := os.Lstat(wd + "/mnt/manual1/file1")
if err != nil {
t.Fatalf("Lstat failed: %v", err)
}
entries, err := ioutil.ReadDir(wd + "/mnt")
if err != nil {
t.Fatalf("ReadDir failed: %v", err)
}
if len(entries) != 3 {
t.Error("readdir mismatch", entries)
}
err = os.Remove(wd + "/mnt/config/manual1")
if err != nil {
t.Fatalf("Remove failed: %v", err)
}
scan := wd + "/mnt/config/" + _SCAN_CONFIG
err = ioutil.WriteFile(scan, []byte("something"), 0644)
if err != nil {
t.Error("error writing:", err)
}
// If FUSE supports invalid inode notifications we expect this node to be gone. Otherwise we'll just make sure that it's not reachable.
if server.KernelSettings().SupportsNotify(fuse.NOTIFY_INVAL_INODE) {
fi, _ = os.Lstat(wd + "/mnt/manual1")
if fi != nil {
t.Error("Should not have file:", fi)
}
} else {
entries, err = ioutil.ReadDir(wd + "/mnt")
if err != nil {
t.Fatalf("ReadDir failed: %v", err)
}
for _, e := range entries {
if e.Name() == "manual1" {
t.Error("Should not have entry: ", e)
}
}
}
_, err = os.Lstat(wd + "/mnt/backing1/file1")
if err != nil {
t.Fatalf("Lstat failed: %v", err)
}
}
func TestDetectSymlinkedDirectories(t *testing.T) {
wd, _, clean := setup(t)
defer clean()
err := os.Mkdir(wd+"/backing1", 0755)
if err != nil {
t.Fatalf("Mkdir failed: %v", err)
}
err = os.Symlink(wd+"/ro", wd+"/backing1/READONLY")
if err != nil {
t.Fatalf("Symlink failed: %v", err)
}
err = os.Symlink(wd+"/backing1", wd+"/store/backing1")
if err != nil {
t.Fatalf("Symlink failed: %v", err)
}
scan := wd + "/mnt/config/" + _SCAN_CONFIG
err = ioutil.WriteFile(scan, []byte("something"), 0644)
if err != nil {
t.Error("error writing:", err)
}
_, err = os.Lstat(wd + "/mnt/backing1")
if err != nil {
t.Fatalf("Lstat failed: %v", err)
}
}
func TestExplicitScan(t *testing.T) {
wd, _, clean := setup(t)
defer clean()
err := os.Mkdir(wd+"/store/backing1", 0755)
if err != nil {
t.Fatalf("Mkdir failed: %v", err)
}
os.Symlink(wd+"/ro", wd+"/store/backing1/READONLY")
if err != nil {
t.Fatalf("Symlink failed: %v", err)
}
fi, _ := os.Lstat(wd + "/mnt/backing1")
if fi != nil {
t.Error("Should not have file:", fi)
}
scan := wd + "/mnt/config/" + _SCAN_CONFIG
_, err = os.Lstat(scan)
if err != nil {
t.Error(".scan_config missing:", err)
}
err = ioutil.WriteFile(scan, []byte("something"), 0644)
if err != nil {
t.Error("error writing:", err)
}
_, err = os.Lstat(wd + "/mnt/backing1")
if err != nil {
t.Error("Should have workspace backing1:", err)
}
}
func TestCreationChecks(t *testing.T) {
wd, _, clean := setup(t)
defer clean()
err := os.Mkdir(wd+"/store/foo", 0755)
if err != nil {
t.Fatalf("Mkdir failed: %v", err)
}
os.Symlink(wd+"/ro", wd+"/store/foo/READONLY")
if err != nil {
t.Fatalf("Symlink failed: %v", err)
}
err = os.Mkdir(wd+"/store/ws2", 0755)
if err != nil {
t.Fatalf("Mkdir failed: %v", err)
}
os.Symlink(wd+"/ro", wd+"/store/ws2/READONLY")
if err != nil {
t.Fatalf("Symlink failed: %v", err)
}
err = os.Symlink(wd+"/store/foo", wd+"/mnt/config/bar")
if err != nil {
t.Fatalf("Symlink failed: %v", err)
}
err = os.Symlink(wd+"/store/foo", wd+"/mnt/config/foo")
code := fuse.ToStatus(err)
if code != fuse.EBUSY {
t.Error("Should return EBUSY", err)
}
err = os.Symlink(wd+"/store/ws2", wd+"/mnt/config/config")
code = fuse.ToStatus(err)
if code != fuse.EINVAL {
t.Error("Should return EINVAL", err)
}
}
unionfs/cachingfs.go deleted 100644 → 0
View file @ 51ae026e
// Copyright 2016 the Go-FUSE Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package unionfs
import (
"fmt"
"log"
"strings"
"time"
"github.com/hanwen/go-fuse/v2/fuse"
"github.com/hanwen/go-fuse/v2/fuse/nodefs"
"github.com/hanwen/go-fuse/v2/fuse/pathfs"
)
const _XATTRSEP = "@XATTR@"
type attrResponse struct {
*fuse.Attr
fuse.Status
}
type xattrResponse struct {
data []byte
fuse.Status
}
type dirResponse struct {
entries []fuse.DirEntry
fuse.Status
}
type linkResponse struct {
linkContent string
fuse.Status
}
// Caches filesystem metadata.
type cachingFileSystem struct {
pathfs.FileSystem
attributes *TimedCache
dirs *TimedCache
links *TimedCache
xattr *TimedCache
}
func readDir(fs pathfs.FileSystem, name string) *dirResponse {
origStream, code := fs.OpenDir(name, nil)
r := &dirResponse{nil, code}
if !code.Ok() {
return r
}
r.entries = origStream
return r
}
func getAttr(fs pathfs.FileSystem, name string) *attrResponse {
a, code := fs.GetAttr(name, nil)
return &attrResponse{
Attr: a,
Status: code,
}
}
func getXAttr(fs pathfs.FileSystem, nameAttr string) *xattrResponse {
ns := strings.SplitN(nameAttr, _XATTRSEP, 2)
a, code := fs.GetXAttr(ns[0], ns[1], nil)
return &xattrResponse{
data: a,
Status: code,
}
}
func readLink(fs pathfs.FileSystem, name string) *linkResponse {
a, code := fs.Readlink(name, nil)
return &linkResponse{
linkContent: a,
Status: code,
}
}
func NewCachingFileSystem(fs pathfs.FileSystem, ttl time.Duration) pathfs.FileSystem {
c := new(cachingFileSystem)
c.FileSystem = fs
c.attributes = NewTimedCache(func(n string) (interface{}, bool) {
a := getAttr(fs, n)
return a, a.Ok()
}, ttl)
c.dirs = NewTimedCache(func(n string) (interface{}, bool) {
d := readDir(fs, n)
return d, d.Ok()
}, ttl)
c.links = NewTimedCache(func(n string) (interface{}, bool) {
l := readLink(fs, n)
return l, l.Ok()
}, ttl)
c.xattr = NewTimedCache(func(n string) (interface{}, bool) {
l := getXAttr(fs, n)
return l, l.Ok()
}, ttl)
return c
}
func (fs *cachingFileSystem) DropCache() {
for _, c := range []*TimedCache{fs.attributes, fs.dirs, fs.links, fs.xattr} {
c.DropAll(nil)
}
}
func (fs *cachingFileSystem) GetAttr(name string, context *fuse.Context) (*fuse.Attr, fuse.Status) {
if name == _DROP_CACHE {
return &fuse.Attr{
Mode: fuse.S_IFREG | 0777,
}, fuse.OK
}
r := fs.attributes.Get(name).(*attrResponse)
return r.Attr, r.Status
}
func (fs *cachingFileSystem) GetXAttr(name string, attr string, context *fuse.Context) ([]byte, fuse.Status) {
key := name + _XATTRSEP + attr
r := fs.xattr.Get(key).(*xattrResponse)
return r.data, r.Status
}
func (fs *cachingFileSystem) Readlink(name string, context *fuse.Context) (string, fuse.Status) {
r := fs.links.Get(name).(*linkResponse)
return r.linkContent, r.Status
}
func (fs *cachingFileSystem) OpenDir(name string, context *fuse.Context) (stream []fuse.DirEntry, status fuse.Status) {
r := fs.dirs.Get(name).(*dirResponse)
return r.entries, r.Status
}
func (fs *cachingFileSystem) String() string {
return fmt.Sprintf("cachingFileSystem(%v)", fs.FileSystem)
}
func (fs *cachingFileSystem) Open(name string, flags uint32, context *fuse.Context) (f nodefs.File, status fuse.Status) {
if flags&fuse.O_ANYWRITE != 0 && name == _DROP_CACHE {
log.Println("Dropping cache for", fs)
fs.DropCache()
}
return fs.FileSystem.Open(name, flags, context)
}
unionfs/cachingfs_test.go deleted 100644 → 0
View file @ 51ae026e
// Copyright 2016 the Go-FUSE Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package unionfs
import (
"os"
"syscall"
"testing"
"github.com/hanwen/go-fuse/v2/fuse"
"github.com/hanwen/go-fuse/v2/fuse/pathfs"
"github.com/hanwen/go-fuse/v2/internal/testutil"
)
func modeMapEq(m1, m2 map[string]uint32) bool {
if len(m1) != len(m2) {
return false
}
for k, v := range m1 {
val, ok := m2[k]
if !ok || val != v {
return false
}
}
return true
}
func TestCachingFs(t *testing.T) {
wd := testutil.TempDir()
defer os.RemoveAll(wd)
fs := pathfs.NewLoopbackFileSystem(wd)
cfs := NewCachingFileSystem(fs, 0)
os.Mkdir(wd+"/orig", 0755)
fi, code := cfs.GetAttr("orig", nil)
if !code.Ok() {
t.Fatal("GetAttr failure", code)
}
if !fi.IsDir() {
t.Error("unexpected attr", fi)
}
os.Symlink("orig", wd+"/symlink")
val, code := cfs.Readlink("symlink", nil)
if val != "orig" {
t.Error("unexpected readlink", val)
}
if !code.Ok() {
t.Error("code !ok ", code)
}
stream, code := cfs.OpenDir("", nil)
if !code.Ok() {
t.Fatal("Readdir fail", code)
}
results := make(map[string]uint32)
for _, v := range stream {
results[v.Name] = v.Mode &^ 07777
}
expected := map[string]uint32{
"symlink": syscall.S_IFLNK,
"orig": fuse.S_IFDIR,
}
if !modeMapEq(results, expected) {
t.Error("Unexpected readdir result", results, expected)
}
}
unionfs/create.go deleted 100644 → 0
View file @ 51ae026e
// Copyright 2016 the Go-FUSE Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package unionfs
import (
"os"
"github.com/hanwen/go-fuse/v2/fuse/pathfs"
)
func NewUnionFsFromRoots(roots []string, opts *UnionFsOptions, roCaching bool) (pathfs.FileSystem, error) {
fses := make([]pathfs.FileSystem, 0)
for i, r := range roots {
var fs pathfs.FileSystem
fi, err := os.Stat(r)
if err != nil {
return nil, err
}
if fi.IsDir() {
fs = pathfs.NewLoopbackFileSystem(r)
}
if fs == nil {
return nil, err
}
if i > 0 && roCaching {
fs = NewCachingFileSystem(fs, 0)
}
fses = append(fses, fs)
}
return NewUnionFs(fses, *opts)
}
unionfs/dircache.go deleted 100644 → 0
View file @ 51ae026e
// Copyright 2016 the Go-FUSE Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package unionfs
import (
"sync"
"time"
"github.com/hanwen/go-fuse/v2/fuse"
"github.com/hanwen/go-fuse/v2/fuse/pathfs"
)
// newDirnameMap reads the contents of the given directory. On error,
// returns a nil map. This forces reloads in the dirCache until we
// succeed.
func newDirnameMap(fs pathfs.FileSystem, dir string) map[string]struct{} {
stream, code := fs.OpenDir(dir, nil)
if code == fuse.ENOENT {
// The directory not existing is not an error.
return map[string]struct{}{}
}
if !code.Ok() {
return nil
}
result := make(map[string]struct{})
for _, e := range stream {
if e.Mode&fuse.S_IFREG != 0 {
result[e.Name] = struct{}{}
}
}
return result
}
// dirCache caches names in a directory for some time.
//
// If called when the cache is expired, the filenames are read afresh in
// the background.
type dirCache struct {
dir string
ttl time.Duration
fs pathfs.FileSystem
// Protects data below.
lock sync.RWMutex
// If nil, you may call refresh() to schedule a new one.
names map[string]struct{}
updateRunning bool
}
func (c *dirCache) setMap(newMap map[string]struct{}) {
c.lock.Lock()
defer c.lock.Unlock()
c.names = newMap
c.updateRunning = false
_ = time.AfterFunc(c.ttl,
func() { c.DropCache() })
}
func (c *dirCache) DropCache() {
c.lock.Lock()
defer c.lock.Unlock()
c.names = nil
}
// Try to refresh: if another update is already running, do nothing,
// otherwise, read the directory and set it.
func (c *dirCache) maybeRefresh() {
c.lock.Lock()
defer c.lock.Unlock()
if c.updateRunning {
return
}
c.updateRunning = true
go func() {
newmap := newDirnameMap(c.fs, c.dir)
c.setMap(newmap)
}()
}
func (c *dirCache) RemoveEntry(name string) {
c.lock.Lock()
defer c.lock.Unlock()
if c.names == nil {
go c.maybeRefresh()
return
}
delete(c.names, name)
}
func (c *dirCache) AddEntry(name string) {
c.lock.Lock()
defer c.lock.Unlock()
if c.names == nil {
go c.maybeRefresh()
return
}
c.names[name] = struct{}{}
}
func newDirCache(fs pathfs.FileSystem, dir string, ttl time.Duration) *dirCache {
dc := new(dirCache)
dc.dir = dir
dc.fs = fs
dc.ttl = ttl
return dc
}
func (c *dirCache) HasEntry(name string) (mapPresent bool, found bool) {
c.lock.RLock()
defer c.lock.RUnlock()
if c.names == nil {
go c.maybeRefresh()
return false, false
}
_, ok := c.names[name]
return true, ok
}
unionfs/timedcache.go deleted 100644 → 0
View file @ 51ae026e
// Copyright 2016 the Go-FUSE Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package unionfs
import (
"sync"
"time"
)
type cacheEntry struct {
data interface{}
// expiry is the absolute timestamp of the expiry.
expiry time.Time
}
// TimedCache caches the result of fetch() for some time. It is
// thread-safe. Calls of fetch() do no happen inside a critical
// section, so when multiple concurrent Get()s happen for the same
// key, multiple fetch() calls may be issued for the same key.
type TimedCacheFetcher func(name string) (value interface{}, cacheable bool)
type TimedCache struct {
fetch TimedCacheFetcher
// ttl is the duration of the cache.
ttl time.Duration
cacheMapMutex sync.RWMutex
cacheMap map[string]*cacheEntry
PurgeTimer *time.Timer
}
// Creates a new cache with the given TTL. If TTL <= 0, the caching is
// indefinite.
func NewTimedCache(fetcher TimedCacheFetcher, ttl time.Duration) *TimedCache {
l := new(TimedCache)
l.ttl = ttl
l.fetch = fetcher
l.cacheMap = make(map[string]*cacheEntry)
return l
}
func (c *TimedCache) Get(name string) interface{} {
c.cacheMapMutex.RLock()
info, ok := c.cacheMap[name]
c.cacheMapMutex.RUnlock()
valid := ok && (c.ttl <= 0 || info.expiry.After(time.Now()))
if valid {
return info.data
}
return c.GetFresh(name)
}
func (c *TimedCache) Set(name string, val interface{}) {
c.cacheMapMutex.Lock()
defer c.cacheMapMutex.Unlock()
c.cacheMap[name] = &cacheEntry{
data: val,
expiry: time.Now().Add(c.ttl),
}
}
func (c *TimedCache) DropEntry(name string) {
c.cacheMapMutex.Lock()
defer c.cacheMapMutex.Unlock()
delete(c.cacheMap, name)
}
func (c *TimedCache) GetFresh(name string) interface{} {
data, ok := c.fetch(name)
if ok {
c.Set(name, data)
}
return data
}
// Drop all expired entries.
func (c *TimedCache) Purge() {
keys := make([]string, 0, len(c.cacheMap))
now := time.Now()
c.cacheMapMutex.Lock()
defer c.cacheMapMutex.Unlock()
for k, v := range c.cacheMap {
if now.After(v.expiry) {
keys = append(keys, k)
}
}
for _, k := range keys {
delete(c.cacheMap, k)
}
}
func (c *TimedCache) DropAll(names []string) {
c.cacheMapMutex.Lock()
defer c.cacheMapMutex.Unlock()
if names == nil {
c.cacheMap = make(map[string]*cacheEntry, len(c.cacheMap))
} else {
for _, nm := range names {
delete(c.cacheMap, nm)
}
}
}
unionfs/timedcache_test.go deleted 100644 → 0
View file @ 51ae026e
// Copyright 2016 the Go-FUSE Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package unionfs
import (
"testing"
"time"
)
func TestTimedCacheUncacheable(t *testing.T) {
fetchCount := 0
fetch := func(n string) (interface{}, bool) {
fetchCount++
i := int(n[0])
return &i, false
}
cache := NewTimedCache(fetch, 0)
v := cache.Get("n").(*int)
w := cache.Get("n").(*int)
if *v != int('n') || *w != *v {
t.Errorf("value mismatch: got %d, %d want %d", *v, *w, int('n'))
}
if fetchCount != 2 {
t.Fatalf("Should have fetched twice: %d", fetchCount)
}
}
func TestTimedCache(t *testing.T) {
fetchCount := 0
fetch := func(n string) (interface{}, bool) {
fetchCount++
i := int(n[0])
return &i, true
}
// This fails with 1e6 on some Opteron CPUs.
ttl := 100 * time.Millisecond
cache := NewTimedCache(fetch, ttl)
v := cache.Get("n").(*int)
if *v != int('n') {
t.Errorf("value mismatch: got %d, want %d", *v, int('n'))
}
if fetchCount != 1 {
t.Errorf("fetch count mismatch: got %d want 1", fetchCount)
}
// The cache update is async.
time.Sleep(time.Duration(ttl / 10))
w := cache.Get("n")
if v != w {
t.Errorf("Huh, inconsistent: 1st = %v != 2nd = %v", v, w)
}
if fetchCount > 1 {
t.Errorf("fetch count fail: %d > 1", fetchCount)
}
time.Sleep(time.Duration(ttl * 2))
cache.Purge()
w = cache.Get("n")
if fetchCount == 1 {
t.Error("Did not fetch again. Purge unsuccessful?")
}
}
unionfs/unionfs.go deleted 100644 → 0
View file @ 51ae026e
// Copyright 2016 the Go-FUSE Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package unionfs
import (
"crypto/md5"
"fmt"
"log"
"os"
"path"
"path/filepath"
"strings"
"sync"
"syscall"
"time"
"github.com/hanwen/go-fuse/v2/fuse"
"github.com/hanwen/go-fuse/v2/fuse/nodefs"
"github.com/hanwen/go-fuse/v2/fuse/pathfs"
)
func filePathHash(path string) string {
dir, base := filepath.Split(path)
h := md5.New()
h.Write([]byte(dir))
return fmt.Sprintf("%x-%s", h.Sum(nil)[:8], base)
}
/*
UnionFs implements a user-space union file system, which is
stateless but efficient even if the writable branch is on NFS.
Assumptions:
* It uses a list of branches, the first of which (index 0) is thought
to be writable, and the rest read-only.
* It assumes that the number of deleted files is small relative to
the total tree size.
Implementation notes.
* It overlays arbitrary writable FileSystems with any number of
readonly FileSystems.
* Deleting a file will put a file named
/DELETIONS/HASH-OF-FULL-FILENAME into the writable overlay,
containing the full filename itself.
This is optimized for NFS usage: we want to minimize the number of
NFS operations, which are slow. By putting all whiteouts in one
place, we can cheaply fetch the list of all deleted files. Even
without caching on our side, the kernel's negative dentry cache can
answer is-deleted queries quickly.
*/
type unionFS struct {
pathfs.FileSystem
// The same, but as interfaces.
fileSystems []pathfs.FileSystem
// A file-existence cache.
deletionCache *dirCache
// A file -> branch cache.
branchCache *TimedCache
// Map of files to hide.
hiddenFiles map[string]bool
options *UnionFsOptions
nodeFs *pathfs.PathNodeFs
}
type UnionFsOptions struct {
BranchCacheTTL time.Duration
DeletionCacheTTL time.Duration
DeletionDirName string
HiddenFiles []string
}
const (
_DROP_CACHE = ".drop_cache"
)
func NewUnionFs(fileSystems []pathfs.FileSystem, options UnionFsOptions) (pathfs.FileSystem, error) {
g := &unionFS{
options: &options,
fileSystems: fileSystems,
FileSystem: pathfs.NewDefaultFileSystem(),
}
writable := g.fileSystems[0]
code := g.createDeletionStore()
if !code.Ok() {
return nil, fmt.Errorf("could not create deletion path %v: %v", options.DeletionDirName, code)
}
g.deletionCache = newDirCache(writable, options.DeletionDirName, options.DeletionCacheTTL)
g.branchCache = NewTimedCache(
func(n string) (interface{}, bool) { return g.getBranchAttrNoCache(n), true },
options.BranchCacheTTL)
g.hiddenFiles = make(map[string]bool)
for _, name := range options.HiddenFiles {
g.hiddenFiles[name] = true
}
return g, nil
}
func (fs *unionFS) OnMount(nodeFs *pathfs.PathNodeFs) {
fs.nodeFs = nodeFs
}
////////////////
// Deal with all the caches.
// The isDeleted() method tells us if a path has a marker in the deletion store.
// It may return an error code if the store could not be accessed.
func (fs *unionFS) isDeleted(name string) (deleted bool, code fuse.Status) {
marker := fs.deletionPath(name)
haveCache, found := fs.deletionCache.HasEntry(filepath.Base(marker))
if haveCache {
return found, fuse.OK
}
_, code = fs.fileSystems[0].GetAttr(marker, nil)
if code == fuse.OK {
return true, code
}
if code == fuse.ENOENT {
return false, fuse.OK
}
log.Printf("error accessing deletion marker %s: %v", marker, code)
return false, fuse.Status(syscall.EROFS)
}
func (fs *unionFS) createDeletionStore() (code fuse.Status) {
writable := fs.fileSystems[0]
fi, code := writable.GetAttr(fs.options.DeletionDirName, nil)
if code == fuse.ENOENT {
code = writable.Mkdir(fs.options.DeletionDirName, 0755, nil)
if code.Ok() {
fi, code = writable.GetAttr(fs.options.DeletionDirName, nil)
}
}
if !code.Ok() || !fi.IsDir() {
code = fuse.Status(syscall.EROFS)
}
return code
}
func (fs *unionFS) getBranch(name string) branchResult {
name = stripSlash(name)
r := fs.branchCache.Get(name)
return r.(branchResult)
}
func (fs *unionFS) setBranch(name string, r branchResult) {
if !r.valid() {
log.Panicf("entry %q setting illegal branchResult %v", name, r)
}
fs.branchCache.Set(name, r)
}
type branchResult struct {
attr *fuse.Attr
code fuse.Status
branch int
}
func (r *branchResult) valid() bool {
return (r.branch >= 0 && r.attr != nil && r.code.Ok()) ||
(r.branch < 0 && r.attr == nil && !r.code.Ok())
}
func (fs branchResult) String() string {
return fmt.Sprintf("{%v %v branch %d}", fs.attr, fs.code, fs.branch)
}
func (fs *unionFS) getBranchAttrNoCache(name string) branchResult {
name = stripSlash(name)
parent, base := path.Split(name)
parent = stripSlash(parent)
parentBranch := 0
if base != "" {
parentBranch = fs.getBranch(parent).branch
}
for i, fs := range fs.fileSystems {
if i < parentBranch {
continue
}
a, s := fs.GetAttr(name, nil)
if s.Ok() {
if i > 0 {
// Needed to make hardlinks work.
a.Ino = 0
}
return branchResult{
attr: a,
code: s,
branch: i,
}
} else {
if s != fuse.ENOENT {
log.Printf("getattr: %v: Got error %v from branch %v", name, s, i)
}
}
}
return branchResult{nil, fuse.ENOENT, -1}
}
////////////////
// Deletion.
func (fs *unionFS) deletionPath(name string) string {
return filepath.Join(fs.options.DeletionDirName, filePathHash(name))
}
func (fs *unionFS) removeDeletion(name string) {
marker := fs.deletionPath(name)
// os.Remove tries to be smart and issues a Remove() and
// Rmdir() sequentially. We want to skip the 2nd system call,
// so use syscall.Unlink() directly.
code := fs.fileSystems[0].Unlink(marker, nil)
if !code.Ok() && code != fuse.ENOENT {
log.Printf("error unlinking %s: %v", marker, code)
}
// Update in-memory cache as last step, so we avoid caching a
// state from before the storage update.
fs.deletionCache.RemoveEntry(path.Base(marker))
}
func (fs *unionFS) putDeletion(name string) (code fuse.Status) {
code = fs.createDeletionStore()
if !code.Ok() {
return code
}
marker := fs.deletionPath(name)
// Is there a WriteStringToFileOrDie ?
writable := fs.fileSystems[0]
fi, code := writable.GetAttr(marker, nil)
if code.Ok() && fi.Size == uint64(len(name)) {
return fuse.OK
}
var f nodefs.File
if code == fuse.ENOENT {
f, code = writable.Create(marker, uint32(os.O_TRUNC|os.O_WRONLY), 0644, nil)
} else {
writable.Chmod(marker, 0644, nil)
f, code = writable.Open(marker, uint32(os.O_TRUNC|os.O_WRONLY), nil)
}
if !code.Ok() {
log.Printf("could not create deletion file %v: %v", marker, code)
return fuse.EPERM
}
defer f.Release()
ctx := &fuse.Context{Cancel: nil}
defer f.Flush(ctx)
n, code := f.Write([]byte(name), 0, ctx)
if int(n) != len(name) || !code.Ok() {
panic(fmt.Sprintf("Error for writing %v: %v, %v (exp %v) %v", name, marker, n, len(name), code))
}
// Update the in-memory deletion cache as the last step,
// to ensure that the new state stays in memory
fs.deletionCache.AddEntry(path.Base(marker))
return fuse.OK
}
////////////////
// Promotion.
func (fs *unionFS) Promote(name string, srcResult branchResult, context *fuse.Context) (code fuse.Status) {
writable := fs.fileSystems[0]
sourceFs := fs.fileSystems[srcResult.branch]
// Promote directories.
fs.promoteDirsTo(name)
if srcResult.attr.IsRegular() {
code = pathfs.CopyFile(sourceFs, writable, name, name, context)
if code.Ok() {
code = writable.Chmod(name, srcResult.attr.Mode&07777|0200, context)
}
if code.Ok() {
aTime := srcResult.attr.AccessTime()
mTime := srcResult.attr.ModTime()
code = writable.Utimens(name, &aTime, &mTime, context)
}
files := fs.nodeFs.AllFiles(name, 0)
for _, fileWrapper := range files {
if !code.Ok() {
break
}
var uf *unionFsFile
f := fileWrapper.File
for f != nil {
ok := false
uf, ok = f.(*unionFsFile)
if ok {
break
}
f = f.InnerFile()
}
if uf == nil {
panic("no unionFsFile found inside")
}
if uf.layer > 0 {
uf.layer = 0
f := uf.File
uf.File, code = fs.fileSystems[0].Open(name, fileWrapper.OpenFlags, context)
f.Flush(context)
f.Release()
}
}
} else if srcResult.attr.IsSymlink() {
link := ""
link, code = sourceFs.Readlink(name, context)
if !code.Ok() {
log.Println("can't read link in source fs", name)
} else {
code = writable.Symlink(link, name, context)
}
} else if srcResult.attr.IsDir() {
code = writable.Mkdir(name, srcResult.attr.Mode&07777|0200, context)
} else {
log.Println("Unknown file type:", srcResult.attr)
return fuse.ENOSYS
}
if !code.Ok() {
fs.branchCache.GetFresh(name)
return code
} else {
r := fs.getBranch(name)
r.branch = 0
fs.setBranch(name, r)
}
return fuse.OK
}
////////////////////////////////////////////////////////////////
// Below: implement interface for a FileSystem.
func (fs *unionFS) Link(orig string, newName string, context *fuse.Context) (code fuse.Status) {
origResult := fs.getBranch(orig)
code = origResult.code
if code.Ok() && origResult.branch > 0 {
code = fs.Promote(orig, origResult, context)
}
if code.Ok() && origResult.branch > 0 {
// Hairy: for the link to be hooked up to the existing
// inode, PathNodeFs must see a client inode for the
// original. We force a refresh of the attribute (so
// the Ino is filled in.), and then force PathNodeFs
// to see the Inode number.
fs.branchCache.GetFresh(orig)
inode := fs.nodeFs.Node(orig)
var a fuse.Attr
inode.Node().GetAttr(&a, nil, nil)
}
if code.Ok() {
code = fs.promoteDirsTo(newName)
}
if code.Ok() {
code = fs.fileSystems[0].Link(orig, newName, context)
}
if code.Ok() {
fs.removeDeletion(newName)
fs.branchCache.GetFresh(newName)
}
return code
}
func (fs *unionFS) Rmdir(path string, context *fuse.Context) (code fuse.Status) {
r := fs.getBranch(path)
if r.code != fuse.OK {
return r.code
}
if !r.attr.IsDir() {
return fuse.Status(syscall.ENOTDIR)
}
stream, code := fs.OpenDir(path, context)
found := false
for _ = range stream {
found = true
}
if found {
return fuse.Status(syscall.ENOTEMPTY)
}
if r.branch > 0 {
code = fs.putDeletion(path)
return code
}
code = fs.fileSystems[0].Rmdir(path, context)
if code != fuse.OK {
return code
}
r = fs.branchCache.GetFresh(path).(branchResult)
if r.branch > 0 {
code = fs.putDeletion(path)
}
return code
}
func (fs *unionFS) Mkdir(path string, mode uint32, context *fuse.Context) (code fuse.Status) {
deleted, code := fs.isDeleted(path)
if !code.Ok() {
return code
}
if !deleted {
r := fs.getBranch(path)
if r.code != fuse.ENOENT {
return fuse.Status(syscall.EEXIST)
}
}
code = fs.promoteDirsTo(path)
if code.Ok() {
code = fs.fileSystems[0].Mkdir(path, mode, context)
}
if code.Ok() {
fs.removeDeletion(path)
attr := &fuse.Attr{
Mode: fuse.S_IFDIR | mode,
}
fs.setBranch(path, branchResult{attr, fuse.OK, 0})
}
var stream []fuse.DirEntry
stream, code = fs.OpenDir(path, context)
if code.Ok() {
// This shouldn't happen, but let's be safe.
for _, entry := range stream {
fs.putDeletion(filepath.Join(path, entry.Name))
}
}
return code
}
func (fs *unionFS) Symlink(pointedTo string, linkName string, context *fuse.Context) (code fuse.Status) {
code = fs.promoteDirsTo(linkName)
if code.Ok() {
code = fs.fileSystems[0].Symlink(pointedTo, linkName, context)
}
if code.Ok() {
fs.removeDeletion(linkName)
fs.branchCache.GetFresh(linkName)
}
return code
}
func (fs *unionFS) Truncate(path string, size uint64, context *fuse.Context) (code fuse.Status) {
if path == _DROP_CACHE {
return fuse.OK
}
r := fs.getBranch(path)
if r.branch > 0 {
code = fs.Promote(path, r, context)
r.branch = 0
}
if code.Ok() {
code = fs.fileSystems[0].Truncate(path, size, context)
}
if code.Ok() {
newAttr := *r.attr
r.attr = &newAttr
r.attr.Size = size
now := time.Now()
r.attr.SetTimes(nil, &now, &now)
fs.setBranch(path, r)
}
return code
}
func (fs *unionFS) Utimens(name string, atime *time.Time, mtime *time.Time, context *fuse.Context) (code fuse.Status) {
name = stripSlash(name)
r := fs.getBranch(name)
code = r.code
if code.Ok() && r.branch > 0 {
code = fs.Promote(name, r, context)
r.branch = 0
}
if code.Ok() {
code = fs.fileSystems[0].Utimens(name, atime, mtime, context)
}
if code.Ok() {
now := time.Now()
newAttr := *r.attr
r.attr = &newAttr
r.attr.SetTimes(atime, mtime, &now)
fs.setBranch(name, r)
}
return code
}
func (fs *unionFS) Chown(name string, uid uint32, gid uint32, context *fuse.Context) (code fuse.Status) {
name = stripSlash(name)
r := fs.getBranch(name)
if r.attr == nil || r.code != fuse.OK {
return r.code
}
newAttr := *r.attr
r.attr = &newAttr
if os.Geteuid() != 0 {
return fuse.EPERM
}
if r.attr.Uid != uid || r.attr.Gid != gid {
if r.branch > 0 {
code := fs.Promote(name, r, context)
if code != fuse.OK {
return code
}
r.branch = 0
}
fs.fileSystems[0].Chown(name, uid, gid, context)
}
r.attr.Uid = uid
r.attr.Gid = gid
now := time.Now()
r.attr.SetTimes(nil, nil, &now)
fs.setBranch(name, r)
return fuse.OK
}
func (fs *unionFS) Chmod(name string, mode uint32, context *fuse.Context) (code fuse.Status) {
name = stripSlash(name)
r := fs.getBranch(name)
if r.attr == nil {
return r.code
}
newAttr := *r.attr
r.attr = &newAttr
if r.code != fuse.OK {
return r.code
}
permMask := uint32(07777)
// Always be writable.
oldMode := r.attr.Mode & permMask
if oldMode != mode {
if r.branch > 0 {
code := fs.Promote(name, r, context)
if code != fuse.OK {
return code
}
r.branch = 0
}
fs.fileSystems[0].Chmod(name, mode, context)
}
r.attr.Mode = (r.attr.Mode &^ permMask) | mode
now := time.Now()
r.attr.SetTimes(nil, nil, &now)
fs.setBranch(name, r)
return fuse.OK
}
func (fs *unionFS) Access(name string, mode uint32, context *fuse.Context) (code fuse.Status) {
// We always allow writing.
mode = mode &^ fuse.W_OK
if name == "" || name == _DROP_CACHE {
return fuse.OK
}
r := fs.getBranch(name)
if r.branch >= 0 {
return fs.fileSystems[r.branch].Access(name, mode, context)
}
return fuse.ENOENT
}
func (fs *unionFS) Unlink(name string, context *fuse.Context) (code fuse.Status) {
r := fs.getBranch(name)
if r.branch == 0 {
code = fs.fileSystems[0].Unlink(name, context)
if code != fuse.OK {
return code
}
r = fs.branchCache.GetFresh(name).(branchResult)
}
if r.branch > 0 {
// It would be nice to do the putDeletion async.
code = fs.putDeletion(name)
}
return code
}
func (fs *unionFS) Readlink(name string, context *fuse.Context) (out string, code fuse.Status) {
r := fs.getBranch(name)
if r.branch >= 0 {
return fs.fileSystems[r.branch].Readlink(name, context)
}
return "", fuse.ENOENT
}
func stripSlash(fn string) string {
return strings.TrimRight(fn, string(filepath.Separator))
}
func (fs *unionFS) promoteDirsTo(filename string) fuse.Status {
dirName, _ := filepath.Split(filename)
dirName = stripSlash(dirName)
var todo []string
var results []branchResult
for dirName != "" {
r := fs.getBranch(dirName)
if !r.code.Ok() {
log.Println("path component does not exist", filename, dirName)
}
if !r.attr.IsDir() {
log.Println("path component is not a directory.", dirName, r)
return fuse.EPERM
}
if r.branch == 0 {
break
}
todo = append(todo, dirName)
results = append(results, r)
dirName, _ = filepath.Split(dirName)
dirName = stripSlash(dirName)
}
for i := range todo {
j := len(todo) - i - 1
d := todo[j]
r := results[j]
code := fs.fileSystems[0].Mkdir(d, r.attr.Mode&07777|0200, nil)
if code != fuse.OK {
log.Println("Error creating dir leading to path", d, code, fs.fileSystems[0])
return fuse.EPERM
}
aTime := r.attr.AccessTime()
mTime := r.attr.ModTime()
fs.fileSystems[0].Utimens(d, &aTime, &mTime, nil)
r.branch = 0
fs.setBranch(d, r)
}
return fuse.OK
}
func (fs *unionFS) Create(name string, flags uint32, mode uint32, context *fuse.Context) (fuseFile nodefs.File, code fuse.Status) {
writable := fs.fileSystems[0]
code = fs.promoteDirsTo(name)
if code != fuse.OK {
return nil, code
}
fuseFile, code = writable.Create(name, flags, mode, context)
if code.Ok() {
fuseFile = fs.newUnionFsFile(fuseFile, 0)
fs.removeDeletion(name)
now := time.Now()
a := fuse.Attr{
Mode: fuse.S_IFREG | mode,
}
a.SetTimes(nil, &now, &now)
fs.setBranch(name, branchResult{&a, fuse.OK, 0})
}
return fuseFile, code
}
func (fs *unionFS) GetAttr(name string, context *fuse.Context) (a *fuse.Attr, s fuse.Status) {
_, hidden := fs.hiddenFiles[name]
if hidden {
return nil, fuse.ENOENT
}
if name == _DROP_CACHE {
return &fuse.Attr{
Mode: fuse.S_IFREG | 0777,
}, fuse.OK
}
if name == fs.options.DeletionDirName {
return nil, fuse.ENOENT
}
isDel, s := fs.isDeleted(name)
if !s.Ok() {
return nil, s
}
if isDel {
return nil, fuse.ENOENT
}
r := fs.getBranch(name)
if r.branch < 0 {
return nil, fuse.ENOENT
}
fi := *r.attr
// Make everything appear writable.
fi.Mode |= 0200
return &fi, r.code
}
func (fs *unionFS) GetXAttr(name string, attr string, context *fuse.Context) ([]byte, fuse.Status) {
if name == _DROP_CACHE {
return nil, fuse.ENOATTR
}
r := fs.getBranch(name)
if r.branch >= 0 {
return fs.fileSystems[r.branch].GetXAttr(name, attr, context)
}
return nil, fuse.ENOENT
}
func (fs *unionFS) OpenDir(directory string, context *fuse.Context) (stream []fuse.DirEntry, status fuse.Status) {
dirBranch := fs.getBranch(directory)
if dirBranch.branch < 0 {
return nil, fuse.ENOENT
}
// We could try to use the cache, but we have a delay, so
// might as well get the fresh results async.
var wg sync.WaitGroup
var deletions map[string]struct{}
wg.Add(1)
go func() {
deletions = newDirnameMap(fs.fileSystems[0], fs.options.DeletionDirName)
wg.Done()
}()
entries := make([]map[string]uint32, len(fs.fileSystems))
for i := range fs.fileSystems {
entries[i] = make(map[string]uint32)
}
statuses := make([]fuse.Status, len(fs.fileSystems))
for i, l := range fs.fileSystems {
if i >= dirBranch.branch {
wg.Add(1)
go func(j int, pfs pathfs.FileSystem) {
ch, s := pfs.OpenDir(directory, context)
statuses[j] = s
for _, v := range ch {
entries[j][v.Name] = v.Mode
}
wg.Done()
}(i, l)
}
}
wg.Wait()
if deletions == nil {
_, code := fs.fileSystems[0].GetAttr(fs.options.DeletionDirName, context)
if code == fuse.ENOENT {
deletions = map[string]struct{}{}
} else {
return nil, fuse.Status(syscall.EROFS)
}
}
results := entries[0]
// TODO(hanwen): should we do anything with the return
// statuses?
for i, m := range entries {
if statuses[i] != fuse.OK {
continue
}
if i == 0 {
// We don't need to further process the first
// branch: it has no deleted files.
continue
}
for k, v := range m {
_, ok := results[k]
if ok {
continue
}
_, deleted := deletions[filePathHash(filepath.Join(directory, k))]
if !deleted {
results[k] = v
}
}
}
if directory == "" {
delete(results, fs.options.DeletionDirName)
for name, _ := range fs.hiddenFiles {
delete(results, name)
}
}
stream = make([]fuse.DirEntry, 0, len(results))
for k, v := range results {
stream = append(stream, fuse.DirEntry{
Name: k,
Mode: v,
})
}
return stream, fuse.OK
}
// recursivePromote promotes path, and if a directory, everything
// below that directory. It returns a list of all promoted paths, in
// full, including the path itself.
func (fs *unionFS) recursivePromote(path string, pathResult branchResult, context *fuse.Context) (names []string, code fuse.Status) {
names = []string{}
if pathResult.branch > 0 {
code = fs.Promote(path, pathResult, context)
}
if code.Ok() {
names = append(names, path)
}
if code.Ok() && pathResult.attr != nil && pathResult.attr.IsDir() {
var stream []fuse.DirEntry
stream, code = fs.OpenDir(path, context)
for _, e := range stream {
if !code.Ok() {
break
}
subnames := []string{}
p := filepath.Join(path, e.Name)
r := fs.getBranch(p)
subnames, code = fs.recursivePromote(p, r, context)
names = append(names, subnames...)
}
}
if !code.Ok() {
names = nil
}
return names, code
}
func (fs *unionFS) renameDirectory(srcResult branchResult, srcDir string, dstDir string, context *fuse.Context) (code fuse.Status) {
names := []string{}
if code.Ok() {
names, code = fs.recursivePromote(srcDir, srcResult, context)
}
if code.Ok() {
code = fs.promoteDirsTo(dstDir)
}
if code.Ok() {
writable := fs.fileSystems[0]
code = writable.Rename(srcDir, dstDir, context)
}
if code.Ok() {
for _, srcName := range names {
relative := strings.TrimLeft(srcName[len(srcDir):], string(filepath.Separator))
dst := filepath.Join(dstDir, relative)
fs.removeDeletion(dst)
srcResult := fs.getBranch(srcName)
srcResult.branch = 0
fs.setBranch(dst, srcResult)
srcResult = fs.branchCache.GetFresh(srcName).(branchResult)
if srcResult.branch > 0 {
code = fs.putDeletion(srcName)
}
}
}
return code
}
func (fs *unionFS) Rename(src string, dst string, context *fuse.Context) fuse.Status {
srcResult := fs.getBranch(src)
if !srcResult.code.Ok() {
return srcResult.code
}
if srcResult.attr.IsDir() {
return fs.renameDirectory(srcResult, src, dst, context)
}
if srcResult.branch > 0 {
if code := fs.Promote(src, srcResult, context); !code.Ok() {
return code
}
}
if code := fs.promoteDirsTo(dst); !code.Ok() {
return code
}
if code := fs.fileSystems[0].Rename(src, dst, context); !code.Ok() {
return code
}
fs.removeDeletion(dst)
// Rename is racy; avoid racing with unionFsFile.Release().
fs.branchCache.DropEntry(dst)
srcResult = fs.branchCache.GetFresh(src).(branchResult)
if srcResult.branch > 0 {
return fs.putDeletion(src)
}
return fuse.OK
}
func (fs *unionFS) DropBranchCache(names []string) {
fs.branchCache.DropAll(names)
}
func (fs *unionFS) DropDeletionCache() {
fs.deletionCache.DropCache()
}
func (fs *unionFS) DropSubFsCaches() {
for _, fs := range fs.fileSystems {
a, code := fs.GetAttr(_DROP_CACHE, nil)
if code.Ok() && a.IsRegular() {
f, _ := fs.Open(_DROP_CACHE, uint32(os.O_WRONLY), nil)
if f != nil {
f.Flush(&fuse.Context{Cancel: nil})
f.Release()
}
}
}
}
func (fs *unionFS) Open(name string, flags uint32, context *fuse.Context) (fuseFile nodefs.File, status fuse.Status) {
if name == _DROP_CACHE {
if flags&fuse.O_ANYWRITE != 0 {
log.Println("Forced cache drop on", fs)
fs.DropBranchCache(nil)
fs.DropDeletionCache()
fs.DropSubFsCaches()
fs.nodeFs.ForgetClientInodes()
}
return nodefs.NewDevNullFile(), fuse.OK
}
r := fs.getBranch(name)
if r.branch < 0 {
// This should not happen, as a GetAttr() should have
// already verified existence.
log.Println("UnionFs: open of non-existent file:", name)
return nil, fuse.ENOENT
}
if flags&fuse.O_ANYWRITE != 0 && r.branch > 0 {
code := fs.Promote(name, r, context)
if code != fuse.OK {
return nil, code
}
r.branch = 0
now := time.Now()
r.attr.SetTimes(nil, &now, nil)
fs.setBranch(name, r)
}
fuseFile, status = fs.fileSystems[r.branch].Open(name, uint32(flags), context)
if fuseFile != nil {
fuseFile = fs.newUnionFsFile(fuseFile, r.branch)
}
return fuseFile, status
}
func (fs *unionFS) String() string {
names := []string{}
for _, fs := range fs.fileSystems {
names = append(names, fs.String())
}
return fmt.Sprintf("UnionFs(%v)", names)
}
func (fs *unionFS) StatFs(name string) *fuse.StatfsOut {
return fs.fileSystems[0].StatFs("")
}
type unionFsFile struct {
nodefs.File
ufs *unionFS
node *nodefs.Inode
layer int
}
func (fs *unionFsFile) String() string {
return fmt.Sprintf("unionFsFile(%s)", fs.File.String())
}
func (fs *unionFS) newUnionFsFile(f nodefs.File, branch int) *unionFsFile {
return &unionFsFile{
File: f,
ufs: fs,
layer: branch,
}
}
func (fs *unionFsFile) InnerFile() (file nodefs.File) {
return fs.File
}
// We can't hook on Release. Release has no response, so it is not
// ordered wrt any following calls.
func (fs *unionFsFile) Flush(ctx *fuse.Context) (code fuse.Status) {
code = fs.File.Flush(ctx)
path := fs.ufs.nodeFs.Path(fs.node)
fs.ufs.branchCache.GetFresh(path)
return code
}
func (fs *unionFsFile) SetInode(node *nodefs.Inode) {
fs.node = node
}
func (fs *unionFsFile) GetAttr(out *fuse.Attr, ctx *fuse.Context) fuse.Status {
code := fs.File.GetAttr(out, ctx)
if code.Ok() {
out.Mode |= 0200
}
return code
}
unionfs/unionfs_test.go deleted 100644 → 0
View file @ 51ae026e
// Copyright 2016 the Go-FUSE Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package unionfs
import (
"bytes"
"fmt"
"io/ioutil"
"os"
"os/exec"
"path/filepath"
"regexp"
"strconv"
"strings"
"syscall"
"testing"
"time"
"github.com/hanwen/go-fuse/v2/fuse"
"github.com/hanwen/go-fuse/v2/fuse/nodefs"
"github.com/hanwen/go-fuse/v2/fuse/pathfs"
"github.com/hanwen/go-fuse/v2/internal/testutil"
"github.com/hanwen/go-fuse/v2/posixtest"
)
func TestFilePathHash(t *testing.T) {
got := filePathHash("xyz/abc")
want := "34d52a6371ee5c79-abc"
if got != want {
t.Errorf("got %q want %q", got, want)
}
}
var testOpts = UnionFsOptions{
DeletionCacheTTL: entryTTL,
DeletionDirName: "DELETIONS",
BranchCacheTTL: entryTTL,
HiddenFiles: []string{"hidden"},
}
func setRecursiveWritable(t *testing.T, dir string, writable bool) {
err := filepath.Walk(
dir,
func(path string, fi os.FileInfo, err error) error {
var newMode uint32
if writable {
newMode = uint32(fi.Mode().Perm()) | 0200
} else {
newMode = uint32(fi.Mode().Perm()) &^ 0222
}
if fi.Mode()|os.ModeSymlink != 0 {
return nil
}
return os.Chmod(path, os.FileMode(newMode))
})
if err != nil {
t.Fatalf("Walk: %v", err)
}
}
// Creates a temporary dir "wd" with 3 directories:
// mnt ... overlayed (unionfs) mount
// rw .... modifiable data
// ro .... read-only data
func setupUfs(t *testing.T) (wd string, cleanup func()) {
// Make sure system setting does not affect test.
syscall.Umask(0)
wd = testutil.TempDir()
err := os.Mkdir(wd+"/mnt", 0700)
if err != nil {
t.Fatalf("Mkdir: %v", err)
}
err = os.Mkdir(wd+"/rw", 0700)
if err != nil {
t.Fatalf("Mkdir: %v", err)
}
os.Mkdir(wd+"/ro", 0700)
if err != nil {
t.Fatalf("Mkdir: %v", err)
}
fses := []pathfs.FileSystem{
pathfs.NewLoopbackFileSystem(wd + "/rw"),
NewCachingFileSystem(pathfs.NewLoopbackFileSystem(wd+"/ro"), 0),
}
ufs, err := NewUnionFs(fses, testOpts)
if err != nil {
t.Fatalf("NewUnionFs: %v", err)
}
// We configure timeouts are smaller, so we can check for
// UnionFs's cache consistency.
opts := &nodefs.Options{
EntryTimeout: entryTTL / 2,
AttrTimeout: entryTTL / 2,
NegativeTimeout: entryTTL / 2,
PortableInodes: true,
Debug: testutil.VerboseTest(),
LookupKnownChildren: true,
}
pathfs := pathfs.NewPathNodeFs(ufs,
&pathfs.PathNodeFsOptions{ClientInodes: true,
Debug: opts.Debug,
})
state, _, err := nodefs.MountRoot(wd+"/mnt", pathfs.Root(), opts)
if err != nil {
t.Fatalf("MountNodeFileSystem: %v", err)
}
go state.Serve()
state.WaitMount()
return wd, func() {
err := state.Unmount()
if err != nil {
return
}
setRecursiveWritable(t, wd, true)
os.RemoveAll(wd)
}
}
func readFromFile(t *testing.T, path string) string {
b, err := ioutil.ReadFile(path)
if err != nil {
t.Fatalf("ReadFile: %v", err)
}
return string(b)
}
func dirNames(t *testing.T, path string) map[string]bool {
f, err := os.Open(path)
if err != nil {
t.Fatalf("Open: %v", err)
}
result := make(map[string]bool)
names, err := f.Readdirnames(-1)
if err != nil {
t.Fatalf("Readdirnames: %v", err)
}
err = f.Close()
if err != nil {
t.Fatalf("Close: %v", err)
}
for _, nm := range names {
result[nm] = true
}
return result
}
func checkMapEq(t *testing.T, m1, m2 map[string]bool) {
if !mapEq(m1, m2) {
msg := fmt.Sprintf("mismatch: got %v != expect %v", m1, m2)
panic(msg)
}
}
func mapEq(m1, m2 map[string]bool) bool {
if len(m1) != len(m2) {
return false
}
for k, v := range m1 {
val, ok := m2[k]
if !ok || val != v {
return false
}
}
return true
}
func fileExists(path string) bool {
f, err := os.Lstat(path)
return err == nil && f != nil
}
func TestUnionFsAutocreateDeletionDir(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
err := os.Remove(wd + "/rw/DELETIONS")
if err != nil {
t.Fatalf("Remove: %v", err)
}
err = os.Mkdir(wd+"/mnt/dir", 0755)
if err != nil {
t.Fatalf("Mkdir: %v", err)
}
_, err = ioutil.ReadDir(wd + "/mnt/dir")
if err != nil {
t.Fatalf("ReadDir: %v", err)
}
}
func TestUnionFsSymlink(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
posixtest.SymlinkReadlink(t, wd+"/mnt")
}
func TestUnionFsSymlinkPromote(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
err := os.Mkdir(wd+"/ro/subdir", 0755)
if err != nil {
t.Fatalf("Mkdir: %v", err)
}
err = os.Symlink("/foobar", wd+"/mnt/subdir/link")
if err != nil {
t.Fatalf("Symlink: %v", err)
}
}
func TestUnionFsChtimes(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
WriteFile(t, wd+"/ro/file", "a")
err := os.Chtimes(wd+"/ro/file", time.Unix(42, 0), time.Unix(43, 0))
if err != nil {
t.Fatalf("Chtimes: %v", err)
}
err = os.Chtimes(wd+"/mnt/file", time.Unix(82, 0), time.Unix(83, 0))
if err != nil {
t.Fatalf("Chtimes: %v", err)
}
fi, err := os.Lstat(wd + "/mnt/file")
attr := &fuse.Attr{}
attr.FromStat(fuse.ToStatT(fi))
if attr.Atime != 82 || attr.Mtime != 83 {
t.Error("Incorrect timestamp", fi)
}
}
func TestUnionFsChmod(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
ro_fn := wd + "/ro/file"
m_fn := wd + "/mnt/file"
WriteFile(t, ro_fn, "a")
err := os.Chmod(m_fn, 00070)
if err != nil {
t.Fatalf("Chmod: %v", err)
}
fi, err := os.Lstat(m_fn)
if err != nil {
t.Fatalf("Lstat: %v", err)
}
if fi.Mode()&07777 != 00270 {
t.Errorf("Unexpected mode found: %o", uint32(fi.Mode().Perm()))
}
_, err = os.Lstat(wd + "/rw/file")
if err != nil {
t.Errorf("File not promoted")
}
}
func TestUnionFsChown(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
ro_fn := wd + "/ro/file"
m_fn := wd + "/mnt/file"
WriteFile(t, ro_fn, "a")
err := os.Chown(m_fn, 0, 0)
code := fuse.ToStatus(err)
if code != fuse.EPERM {
t.Error("Unexpected error code", code, err)
}
}
func TestUnionFsDelete(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
WriteFile(t, wd+"/ro/file", "a")
_, err := os.Lstat(wd + "/mnt/file")
if err != nil {
t.Fatalf("Lstat: %v", err)
}
err = os.Remove(wd + "/mnt/file")
if err != nil {
t.Fatalf("Remove: %v", err)
}
_, err = os.Lstat(wd + "/mnt/file")
if err == nil {
t.Fatal("should have disappeared.")
}
delPath := wd + "/rw/" + testOpts.DeletionDirName
names := dirNames(t, delPath)
if len(names) != 1 {
t.Fatal("Should have 1 deletion", names)
}
for k := range names {
c, err := ioutil.ReadFile(delPath + "/" + k)
if err != nil {
t.Fatalf("ReadFile: %v", err)
}
if string(c) != "file" {
t.Fatal("content mismatch", string(c))
}
}
}
func TestUnionFsBasic(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
WriteFile(t, wd+"/rw/rw", "a")
WriteFile(t, wd+"/ro/ro1", "a")
WriteFile(t, wd+"/ro/ro2", "b")
names := dirNames(t, wd+"/mnt")
expected := map[string]bool{
"rw": true, "ro1": true, "ro2": true,
}
checkMapEq(t, names, expected)
WriteFile(t, wd+"/mnt/new", "new contents")
if !fileExists(wd + "/rw/new") {
t.Errorf("missing file in rw layer: %s", wd+"/rw/new")
}
contents := readFromFile(t, wd+"/mnt/new")
if contents != "new contents" {
t.Errorf("read mismatch: '%v'", contents)
}
WriteFile(t, wd+"/mnt/ro1", "promote me")
if !fileExists(wd + "/rw/ro1") {
t.Errorf("missing file in rw layer: %s", wd+"/mnt/ro1")
}
err := os.Remove(wd + "/mnt/new")
if err != nil {
t.Fatalf("Remove: %v", err)
}
names = dirNames(t, wd+"/mnt")
checkMapEq(t, names, map[string]bool{
"rw": true, "ro1": true, "ro2": true,
})
names = dirNames(t, wd+"/rw")
checkMapEq(t, names, map[string]bool{
testOpts.DeletionDirName: true,
"rw": true, "ro1": true,
})
names = dirNames(t, wd+"/rw/"+testOpts.DeletionDirName)
if len(names) != 0 {
t.Errorf("Expected 0 entry in %v", names)
}
err = os.Remove(wd + "/mnt/ro1")
if err != nil {
t.Fatalf("Remove: %v", err)
}
names = dirNames(t, wd+"/mnt")
checkMapEq(t, names, map[string]bool{
"rw": true, "ro2": true,
})
names = dirNames(t, wd+"/rw")
checkMapEq(t, names, map[string]bool{
"rw": true, testOpts.DeletionDirName: true,
})
names = dirNames(t, wd+"/rw/"+testOpts.DeletionDirName)
if len(names) != 1 {
t.Errorf("Expected 1 entry in %v", names)
}
}
func TestUnionFsPromote(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
err := os.Mkdir(wd+"/ro/subdir", 0755)
if err != nil {
t.Fatalf("Mkdir: %v", err)
}
WriteFile(t, wd+"/ro/subdir/file", "content")
WriteFile(t, wd+"/mnt/subdir/file", "other-content")
}
func TestUnionFsCreate(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
err := os.MkdirAll(wd+"/ro/subdir/sub2", 0755)
if err != nil {
t.Fatalf("MkdirAll: %v", err)
}
WriteFile(t, wd+"/mnt/subdir/sub2/file", "other-content")
_, err = os.Lstat(wd + "/mnt/subdir/sub2/file")
if err != nil {
t.Fatalf("Lstat: %v", err)
}
}
func TestUnionFsOpenUndeletes(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
WriteFile(t, wd+"/ro/file", "X")
err := os.Remove(wd + "/mnt/file")
if err != nil {
t.Fatalf("Remove: %v", err)
}
WriteFile(t, wd+"/mnt/file", "X")
_, err = os.Lstat(wd + "/mnt/file")
if err != nil {
t.Fatalf("Lstat: %v", err)
}
}
func TestUnionFsMkdir(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
posixtest.MkdirRmdir(t, wd+"/mnt")
}
func TestUnionFsMkdirPromote(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
dirname := wd + "/ro/subdir/subdir2"
err := os.MkdirAll(dirname, 0755)
if err != nil {
t.Fatalf("MkdirAll: %v", err)
}
err = os.Mkdir(wd+"/mnt/subdir/subdir2/dir3", 0755)
if err != nil {
t.Fatalf("Mkdir: %v", err)
}
fi, _ := os.Lstat(wd + "/rw/subdir/subdir2/dir3")
if err != nil {
t.Fatalf("Lstat: %v", err)
}
if fi == nil || !fi.IsDir() {
t.Error("is not a directory: ", fi)
}
}
func TestUnionFsRmdirMkdir(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
err := os.Mkdir(wd+"/ro/subdir", 0755)
if err != nil {
t.Fatalf("Mkdir: %v", err)
}
dirname := wd + "/mnt/subdir"
err = os.Remove(dirname)
if err != nil {
t.Fatalf("Remove: %v", err)
}
err = os.Mkdir(dirname, 0755)
if err != nil {
t.Fatalf("Mkdir: %v", err)
}
}
func TestUnionFsRename(t *testing.T) {
type Config struct {
f1_ro bool
f1_rw bool
f2_ro bool
f2_rw bool
}
configs := make([]Config, 0)
for i := 0; i < 16; i++ {
c := Config{i&0x1 != 0, i&0x2 != 0, i&0x4 != 0, i&0x8 != 0}
if !(c.f1_ro || c.f1_rw) {
continue
}
configs = append(configs, c)
}
for i, c := range configs {
t.Run(fmt.Sprintf("config %d", i), func(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
if c.f1_ro {
WriteFile(t, wd+"/ro/file1", "c1")
}
if c.f1_rw {
WriteFile(t, wd+"/rw/file1", "c2")
}
if c.f2_ro {
WriteFile(t, wd+"/ro/file2", "c3")
}
if c.f2_rw {
WriteFile(t, wd+"/rw/file2", "c4")
}
err := os.Rename(wd+"/mnt/file1", wd+"/mnt/file2")
if err != nil {
t.Fatalf("Rename: %v", err)
}
_, err = os.Lstat(wd + "/mnt/file1")
if err == nil {
t.Errorf("Should have lost file1")
}
_, err = os.Lstat(wd + "/mnt/file2")
if err != nil {
t.Errorf("Should have gotten file2: %v", err)
}
err = os.Rename(wd+"/mnt/file2", wd+"/mnt/file1")
if err != nil {
t.Fatalf("Rename: %v", err)
}
_, err = os.Lstat(wd + "/mnt/file2")
if err == nil {
t.Errorf("Should have lost file2")
}
_, err = os.Lstat(wd + "/mnt/file1")
if err != nil {
t.Errorf("Should have gotten file1: %v", err)
}
})
}
}
func TestUnionFsRenameDirBasic(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
err := os.MkdirAll(wd+"/ro/dir/subdir", 0755)
if err != nil {
t.Fatalf("MkdirAll: %v", err)
}
err = os.Rename(wd+"/mnt/dir", wd+"/mnt/renamed")
if err != nil {
t.Fatalf("Rename: %v", err)
}
if fi, _ := os.Lstat(wd + "/mnt/dir"); fi != nil {
t.Fatalf("%s/mnt/dir should have disappeared: %v", wd, fi)
}
if fi, _ := os.Lstat(wd + "/mnt/renamed"); fi == nil || !fi.IsDir() {
t.Fatalf("%s/mnt/renamed should be directory: %v", wd, fi)
}
entries, err := ioutil.ReadDir(wd + "/mnt/renamed")
if err != nil || len(entries) != 1 || entries[0].Name() != "subdir" {
t.Errorf("readdir(%s/mnt/renamed) should have one entry: %v, err %v", wd, entries, err)
}
if err = os.Mkdir(wd+"/mnt/dir", 0755); err != nil {
t.Errorf("mkdir should succeed %v", err)
}
}
func TestUnionFsRenameDirAllSourcesGone(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
err := os.MkdirAll(wd+"/ro/dir", 0755)
if err != nil {
t.Fatalf("MkdirAll: %v", err)
}
err = ioutil.WriteFile(wd+"/ro/dir/file.txt", []byte{42}, 0644)
if err != nil {
t.Fatalf("WriteFile: %v", err)
}
setRecursiveWritable(t, wd+"/ro", false)
err = os.Rename(wd+"/mnt/dir", wd+"/mnt/renamed")
if err != nil {
t.Fatalf("Rename: %v", err)
}
names := dirNames(t, wd+"/rw/"+testOpts.DeletionDirName)
if len(names) != 2 {
t.Errorf("Expected 2 entries in %v", names)
}
}
func TestUnionFsRenameDirWithDeletions(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
err := os.MkdirAll(wd+"/ro/dir/subdir", 0755)
if err != nil {
t.Fatalf("MkdirAll: %v", err)
}
err = ioutil.WriteFile(wd+"/ro/dir/file.txt", []byte{42}, 0644)
if err != nil {
t.Fatalf("WriteFile: %v", err)
}
err = ioutil.WriteFile(wd+"/ro/dir/subdir/file.txt", []byte{42}, 0644)
if err != nil {
t.Fatalf("WriteFile: %v", err)
}
setRecursiveWritable(t, wd+"/ro", false)
if fi, _ := os.Lstat(wd + "/mnt/dir/subdir/file.txt"); fi == nil || fi.Mode()&os.ModeType != 0 {
t.Fatalf("%s/mnt/dir/subdir/file.txt should be file: %v", wd, fi)
}
err = os.Remove(wd + "/mnt/dir/file.txt")
if err != nil {
t.Fatalf("Remove: %v", err)
}
err = os.Rename(wd+"/mnt/dir", wd+"/mnt/renamed")
if err != nil {
t.Fatalf("Rename: %v", err)
}
if fi, _ := os.Lstat(wd + "/mnt/dir/subdir/file.txt"); fi != nil {
t.Fatalf("%s/mnt/dir/subdir/file.txt should have disappeared: %v", wd, fi)
}
if fi, _ := os.Lstat(wd + "/mnt/dir"); fi != nil {
t.Fatalf("%s/mnt/dir should have disappeared: %v", wd, fi)
}
if fi, _ := os.Lstat(wd + "/mnt/renamed"); fi == nil || !fi.IsDir() {
t.Fatalf("%s/mnt/renamed should be directory: %v", wd, fi)
}
if fi, _ := os.Lstat(wd + "/mnt/renamed/file.txt"); fi != nil {
t.Fatalf("%s/mnt/renamed/file.txt should have disappeared %#v", wd, fi)
}
if err = os.Mkdir(wd+"/mnt/dir", 0755); err != nil {
t.Errorf("mkdir should succeed %v", err)
}
if fi, _ := os.Lstat(wd + "/mnt/dir/subdir"); fi != nil {
t.Fatalf("%s/mnt/dir/subdir should have disappeared %#v", wd, fi)
}
}
func TestUnionFsRenameSymlink(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
err := os.Symlink("linktarget", wd+"/ro/link")
if err != nil {
t.Fatalf("Symlink: %v", err)
}
err = os.Rename(wd+"/mnt/link", wd+"/mnt/renamed")
if err != nil {
t.Fatalf("Rename: %v", err)
}
if fi, _ := os.Lstat(wd + "/mnt/link"); fi != nil {
t.Fatalf("%s/mnt/link should have disappeared: %v", wd, fi)
}
if fi, _ := os.Lstat(wd + "/mnt/renamed"); fi == nil || fi.Mode()&os.ModeSymlink == 0 {
t.Fatalf("%s/mnt/renamed should be link: %v", wd, fi)
}
if link, err := os.Readlink(wd + "/mnt/renamed"); err != nil || link != "linktarget" {
t.Fatalf("readlink(%s/mnt/renamed) should point to 'linktarget': %v, err %v", wd, link, err)
}
}
func TestUnionFsWritableDir(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
dirname := wd + "/ro/subdir"
err := os.Mkdir(dirname, 0555)
if err != nil {
t.Fatalf("Mkdir: %v", err)
}
setRecursiveWritable(t, wd+"/ro", false)
fi, err := os.Lstat(wd + "/mnt/subdir")
if err != nil {
t.Fatalf("Lstat: %v", err)
}
if fi.Mode().Perm()&0222 == 0 {
t.Errorf("unexpected permission %o", fi.Mode().Perm())
}
}
func TestUnionFsWriteAccess(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
fn := wd + "/ro/file"
// No write perms.
err := ioutil.WriteFile(fn, []byte("foo"), 0444)
if err != nil {
t.Fatalf("WriteFile: %v", err)
}
setRecursiveWritable(t, wd+"/ro", false)
err = syscall.Access(wd+"/mnt/file", fuse.W_OK)
if err != nil {
if err != nil {
t.Fatalf("Access: %v", err)
}
}
}
func TestUnionFsLink(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
content := "blabla"
fn := wd + "/ro/file"
err := ioutil.WriteFile(fn, []byte(content), 0666)
if err != nil {
t.Fatalf("WriteFile: %v", err)
}
setRecursiveWritable(t, wd+"/ro", false)
err = os.Link(wd+"/mnt/file", wd+"/mnt/linked")
if err != nil {
t.Fatalf("Link: %v", err)
}
fi2, err := os.Lstat(wd + "/mnt/linked")
if err != nil {
t.Fatalf("Lstat: %v", err)
}
fi1, err := os.Lstat(wd + "/mnt/file")
if err != nil {
t.Fatalf("Lstat: %v", err)
}
s1 := fuse.ToStatT(fi1)
s2 := fuse.ToStatT(fi2)
if s1.Ino != s2.Ino {
t.Errorf("inode numbers should be equal for linked files %v, %v", s1.Ino, s2.Ino)
}
c, err := ioutil.ReadFile(wd + "/mnt/linked")
if string(c) != content {
t.Errorf("content mismatch got %q want %q", string(c), content)
}
}
func TestUnionFsTruncate(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
WriteFile(t, wd+"/ro/file", "hello")
setRecursiveWritable(t, wd+"/ro", false)
os.Truncate(wd+"/mnt/file", 2)
content := readFromFile(t, wd+"/mnt/file")
if content != "he" {
t.Errorf("unexpected content %v", content)
}
content2 := readFromFile(t, wd+"/rw/file")
if content2 != content {
t.Errorf("unexpected rw content %v", content2)
}
}
func TestUnionFsCopyChmod(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
contents := "hello"
fn := wd + "/mnt/y"
err := ioutil.WriteFile(fn, []byte(contents), 0644)
if err != nil {
t.Fatalf("WriteFile(%v): %v", fn, err)
}
err = os.Chmod(fn, 0755)
if err != nil {
t.Fatalf("Chmod(%v): %v", fn, err)
}
fi, err := os.Lstat(fn)
if err != nil {
t.Fatalf("Lstat(%v): %v", fn, err)
}
if fi.Mode()&0111 == 0 {
t.Errorf("Lstat(%v): got mode %o, want some +x bit", fn, fi.Mode())
}
time.Sleep(entryTTL)
fi, err = os.Lstat(fn)
if err != nil {
t.Fatalf("Lstat(%v) after sleep: %v", fn, err)
}
if fi.Mode()&0111 == 0 {
t.Errorf("Lstat(%v) after sleep: mode %o", fn, fi.Mode())
}
}
func abs(dt int64) int64 {
if dt >= 0 {
return dt
}
return -dt
}
func TestUnionFsTruncateTimestamp(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
contents := "hello"
fn := wd + "/mnt/y"
err := ioutil.WriteFile(fn, []byte(contents), 0644)
if err != nil {
t.Fatalf("WriteFile(%v): %v", fn, err)
}
time.Sleep(200 * time.Millisecond)
truncTs := time.Now()
err = os.Truncate(fn, 3)
if err != nil {
t.Fatalf("Truncate(%v): %v", fn, err)
}
fi, err := os.Lstat(fn)
if err != nil {
t.Fatalf("Lstat(%v): %v", fn, err)
}
if truncTs.Sub(fi.ModTime()) > 100*time.Millisecond {
t.Errorf("after Truncate: got TS %v, want %v", fi.ModTime(), truncTs)
}
}
func TestUnionFsRemoveAll(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
err := os.MkdirAll(wd+"/ro/dir/subdir", 0755)
if err != nil {
t.Fatalf("MkdirAll: %v", err)
}
contents := "hello"
fn := wd + "/ro/dir/subdir/y"
err = ioutil.WriteFile(fn, []byte(contents), 0644)
if err != nil {
t.Fatalf("WriteFile: %v", err)
}
setRecursiveWritable(t, wd+"/ro", false)
err = os.RemoveAll(wd + "/mnt/dir")
if err != nil {
t.Error("Should delete all")
}
for _, f := range []string{"dir/subdir/y", "dir/subdir", "dir"} {
if fi, _ := os.Lstat(filepath.Join(wd, "mount", f)); fi != nil {
t.Errorf("file %s should have disappeared: %v", f, fi)
}
}
names, err := Readdirnames(wd + "/rw/DELETIONS")
if err != nil {
t.Fatalf("Readdirnames: %v", err)
}
if len(names) != 3 {
t.Fatal("unexpected names", names)
}
}
func ProgramVersion(bin string) (major, minor int64, err error) {
cmd := exec.Command(bin, "--version")
buf := &bytes.Buffer{}
cmd.Stdout = buf
if err := cmd.Run(); err != nil {
return 0, 0, err
}
lines := strings.Split(buf.String(), "\n")
if len(lines) < 1 {
return 0, 0, fmt.Errorf("no output")
}
matches := regexp.MustCompile(".* ([0-9]+)\\.([0-9]+)").FindStringSubmatch(lines[0])
if matches == nil {
return 0, 0, fmt.Errorf("no match for %q", lines[0])
}
major, err = strconv.ParseInt(matches[1], 10, 64)
if err != nil {
return 0, 0, err
}
minor, err = strconv.ParseInt(matches[2], 10, 64)
if err != nil {
return 0, 0, err
}
return major, minor, nil
}
func TestUnionFsRmRf(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
err := os.MkdirAll(wd+"/ro/dir/subdir", 0755)
if err != nil {
t.Fatalf("MkdirAll: %v", err)
}
contents := "hello"
fn := wd + "/ro/dir/subdir/y"
err = ioutil.WriteFile(fn, []byte(contents), 0644)
if err != nil {
t.Fatalf("WriteFile: %v", err)
}
setRecursiveWritable(t, wd+"/ro", false)
bin, err := exec.LookPath("rm")
if err != nil {
t.Fatalf("LookPath: %v", err)
}
maj, min, err := ProgramVersion(bin)
if err != nil {
t.Logf("ProgramVersion: %v", err)
}
if maj < 8 { // assuming GNU coreutils.
t.Skipf("Skipping test; GNU rm %d.%d is not POSIX compliant.", maj, min)
}
names, _ := Readdirnames(wd + "/mnt/dir")
t.Logf("Contents of %s/mnt/dir: %s", wd, strings.Join(names, ", "))
cmd := exec.Command(bin, "-rf", wd+"/mnt/dir")
err = cmd.Run()
if err != nil {
t.Fatal("rm -rf returned error:", err)
}
for _, f := range []string{"dir/subdir/y", "dir/subdir", "dir"} {
if fi, _ := os.Lstat(filepath.Join(wd, "mount", f)); fi != nil {
t.Errorf("file %s should have disappeared: %v", f, fi)
}
}
names, err = Readdirnames(wd + "/rw/DELETIONS")
if err != nil {
t.Fatalf("Readdirnames: %v", err)
}
if len(names) != 3 {
t.Fatal("unexpected names", names)
}
}
func Readdirnames(dir string) ([]string, error) {
f, err := os.Open(dir)
if err != nil {
return nil, err
}
defer f.Close()
return f.Readdirnames(-1)
}
func TestUnionFsDropDeletionCache(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
err := ioutil.WriteFile(wd+"/ro/file", []byte("bla"), 0644)
if err != nil {
t.Fatalf("WriteFile: %v", err)
}
setRecursiveWritable(t, wd+"/ro", false)
_, err = os.Lstat(wd + "/mnt/file")
if err != nil {
t.Fatalf("Lstat: %v", err)
}
err = os.Remove(wd + "/mnt/file")
if err != nil {
t.Fatalf("Remove: %v", err)
}
fi, _ := os.Lstat(wd + "/mnt/file")
if fi != nil {
t.Fatal("Lstat() should have failed", fi)
}
names, err := Readdirnames(wd + "/rw/DELETIONS")
if err != nil {
t.Fatalf("Readdirnames: %v", err)
}
if len(names) != 1 {
t.Fatal("unexpected names", names)
}
os.Remove(wd + "/rw/DELETIONS/" + names[0])
fi, _ = os.Lstat(wd + "/mnt/file")
if fi != nil {
t.Fatal("Lstat() should have failed", fi)
}
// Expire kernel entry.
time.Sleep((6 * entryTTL) / 10)
err = ioutil.WriteFile(wd+"/mnt/.drop_cache", []byte(""), 0644)
if err != nil {
t.Fatalf("WriteFile: %v", err)
}
_, err = os.Lstat(wd + "/mnt/file")
if err != nil {
t.Fatal("Lstat() should have succeeded", err)
}
}
func TestUnionFsDropCache(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
err := ioutil.WriteFile(wd+"/ro/file", []byte("bla"), 0644)
if err != nil {
t.Fatalf("WriteFile: %v", err)
}
_, err = os.Lstat(wd + "/mnt/.drop_cache")
if err != nil {
t.Fatalf("Lstat: %v", err)
}
names, err := Readdirnames(wd + "/mnt")
if err != nil {
t.Fatalf("Readdirnames: %v", err)
}
if len(names) != 1 || names[0] != "file" {
t.Fatal("unexpected names", names)
}
err = ioutil.WriteFile(wd+"/ro/file2", []byte("blabla"), 0644)
names2, err := Readdirnames(wd + "/mnt")
if err != nil {
t.Fatalf("Readdirnames: %v", err)
}
if len(names2) != len(names) {
t.Fatal("mismatch", names2)
}
err = ioutil.WriteFile(wd+"/mnt/.drop_cache", []byte("does not matter"), 0644)
if err != nil {
t.Fatalf("WriteFile: %v", err)
}
names2, err = Readdirnames(wd + "/mnt")
if len(names2) != 2 {
t.Fatal("mismatch 2", names2)
}
}
type disappearingFS struct {
pathfs.FileSystem
normal pathfs.FileSystem
nop pathfs.FileSystem
visible bool
visibleChan chan bool
}
func (d *disappearingFS) fs() pathfs.FileSystem {
select {
case v := <-d.visibleChan:
d.visible = v
if v {
d.FileSystem = d.normal
} else {
d.FileSystem = d.nop
}
default:
}
return d.FileSystem
}
func (d *disappearingFS) GetAttr(name string, context *fuse.Context) (a *fuse.Attr, s fuse.Status) {
return d.fs().GetAttr(name, context)
}
func (d *disappearingFS) OpenDir(name string, context *fuse.Context) ([]fuse.DirEntry, fuse.Status) {
return d.fs().OpenDir(name, context)
}
func newDisappearingFS(fs, nop pathfs.FileSystem) *disappearingFS {
return &disappearingFS{
visibleChan: make(chan bool, 1),
visible: true,
normal: fs,
nop: nop,
FileSystem: fs,
}
}
func TestUnionFsDisappearing(t *testing.T) {
// This init is like setupUfs, but we want access to the
// writable Fs.
wd := testutil.TempDir()
defer os.RemoveAll(wd)
err := os.Mkdir(wd+"/mnt", 0700)
if err != nil {
t.Fatalf("Mkdir: %v", err)
}
err = os.Mkdir(wd+"/rw", 0700)
if err != nil {
t.Fatalf("Mkdir: %v", err)
}
os.Mkdir(wd+"/ro", 0700)
if err != nil {
t.Fatalf("Mkdir: %v", err)
}
wrFs := newDisappearingFS(pathfs.NewLoopbackFileSystem(wd+"/rw"),
pathfs.NewLoopbackFileSystem("/dev/null"))
var fses []pathfs.FileSystem
fses = append(fses, pathfs.NewLockingFileSystem(wrFs))
fses = append(fses, pathfs.NewLoopbackFileSystem(wd+"/ro"))
ufs, err := NewUnionFs(fses, testOpts)
if err != nil {
t.Fatalf("NewUnionFs: %v", err)
}
opts := &nodefs.Options{
EntryTimeout: entryTTL,
AttrTimeout: entryTTL,
NegativeTimeout: entryTTL,
Debug: testutil.VerboseTest(),
LookupKnownChildren: true,
}
nfs := pathfs.NewPathNodeFs(ufs, nil)
state, _, err := nodefs.MountRoot(wd+"/mnt", nfs.Root(), opts)
if err != nil {
t.Fatalf("MountNodeFileSystem: %v", err)
}
defer state.Unmount()
go state.Serve()
state.WaitMount()
err = ioutil.WriteFile(wd+"/ro/file", []byte("blabla"), 0644)
if err != nil {
t.Fatalf("WriteFile: %v", err)
}
setRecursiveWritable(t, wd+"/ro", false)
err = os.Remove(wd + "/mnt/file")
if err != nil {
t.Fatalf("Remove: %v", err)
}
wrFs.visibleChan <- false
time.Sleep((3 * entryTTL) / 2)
_, err = ioutil.ReadDir(wd + "/mnt")
if err == nil {
t.Fatal("Readdir should have failed")
}
err = ioutil.WriteFile(wd+"/mnt/file2", []byte("blabla"), 0644)
if err == nil {
t.Fatal("write should have failed")
}
// Wait for the caches to purge, and then restore.
time.Sleep((3 * entryTTL) / 2)
wrFs.visibleChan <- true
_, err = ioutil.ReadDir(wd + "/mnt")
if err != nil {
t.Fatal("Readdir should succeed", err)
}
err = ioutil.WriteFile(wd+"/mnt/file2", []byte("blabla"), 0644)
if err != nil {
t.Fatal("write should succeed", err)
}
}
func TestUnionFsDeletedGetAttr(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
err := ioutil.WriteFile(wd+"/ro/file", []byte("blabla"), 0644)
if err != nil {
t.Fatalf("WriteFile: %v", err)
}
setRecursiveWritable(t, wd+"/ro", false)
f, err := os.Open(wd + "/mnt/file")
if err != nil {
t.Fatalf("Open: %v", err)
}
defer f.Close()
err = os.Remove(wd + "/mnt/file")
if err != nil {
t.Fatalf("Remove: %v", err)
}
if fi, err := f.Stat(); err != nil || fi.Mode()&os.ModeType != 0 {
t.Fatalf("stat returned error or non-file: %v %v", err, fi)
}
}
func TestUnionFsDoubleOpen(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
err := ioutil.WriteFile(wd+"/ro/file", []byte("blablabla"), 0644)
if err != nil {
t.Fatalf("WriteFile: %v", err)
}
setRecursiveWritable(t, wd+"/ro", false)
roFile, err := os.Open(wd + "/mnt/file")
if err != nil {
t.Fatalf("Open: %v", err)
}
defer roFile.Close()
rwFile, err := os.OpenFile(wd+"/mnt/file", os.O_WRONLY|os.O_TRUNC, 0666)
if err != nil {
t.Fatalf("OpenFile: %v", err)
}
defer rwFile.Close()
output, err := ioutil.ReadAll(roFile)
if err != nil {
t.Fatalf("ReadAll: %v", err)
}
if len(output) != 0 {
t.Errorf("After r/w truncation, r/o file should be empty too: %q", string(output))
}
want := "hello"
_, err = rwFile.Write([]byte(want))
if err != nil {
t.Fatalf("Write: %v", err)
}
b := make([]byte, 100)
roFile.Seek(0, 0)
n, err := roFile.Read(b)
if err != nil {
t.Fatalf("Read: %v", err)
}
b = b[:n]
if string(b) != "hello" {
t.Errorf("r/w and r/o file are not synchronized: got %q want %q", string(b), want)
}
}
func TestUnionFsStatFs(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
s1 := syscall.Statfs_t{}
err := syscall.Statfs(wd+"/mnt", &s1)
if err != nil {
t.Fatal("statfs mnt", err)
}
if s1.Bsize == 0 {
t.Fatal("Expect blocksize > 0")
}
}
func TestUnionFsFlushSize(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
fn := wd + "/mnt/file"
f, err := os.OpenFile(fn, os.O_WRONLY|os.O_CREATE, 0644)
if err != nil {
t.Fatalf("OpenFile: %v", err)
}
fi, err := f.Stat()
if err != nil {
t.Fatalf("Stat: %v", err)
}
n, err := f.Write([]byte("hello"))
if err != nil {
t.Fatalf("Write: %v", err)
}
f.Close()
fi, err = os.Lstat(fn)
if err != nil {
t.Fatalf("Lstat: %v", err)
}
if fi.Size() != int64(n) {
t.Errorf("got %d from Stat().Size, want %d", fi.Size(), n)
}
}
func TestUnionFsFlushRename(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
err := ioutil.WriteFile(wd+"/mnt/file", []byte("x"), 0644)
fn := wd + "/mnt/tmp"
f, err := os.OpenFile(fn, os.O_WRONLY|os.O_CREATE, 0644)
if err != nil {
t.Fatalf("OpenFile: %v", err)
}
fi, err := f.Stat()
if err != nil {
t.Fatalf("Stat: %v", err)
}
n, err := f.Write([]byte("hello"))
if err != nil {
t.Fatalf("Write: %v", err)
}
f.Close()
dst := wd + "/mnt/file"
err = os.Rename(fn, dst)
if err != nil {
t.Fatalf("Rename: %v", err)
}
fi, err = os.Lstat(dst)
if err != nil {
t.Fatalf("Lstat: %v", err)
}
if fi.Size() != int64(n) {
t.Errorf("got %d from Stat().Size, want %d", fi.Size(), n)
}
}
func TestUnionFsTruncGetAttr(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
c := []byte("hello")
f, err := os.OpenFile(wd+"/mnt/file", os.O_CREATE|os.O_RDWR|os.O_TRUNC, 0644)
if err != nil {
t.Fatalf("OpenFile: %v", err)
}
_, err = f.Write(c)
if err != nil {
t.Fatalf("Write: %v", err)
}
err = f.Close()
if err != nil {
t.Fatalf("Close: %v", err)
}
fi, err := os.Lstat(wd + "/mnt/file")
if fi.Size() != int64(len(c)) {
t.Fatalf("Length mismatch got %d want %d", fi.Size(), len(c))
}
}
func TestUnionFsPromoteDirTimeStamp(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
err := os.Mkdir(wd+"/ro/subdir", 0750)
if err != nil {
t.Fatalf("Mkdir: %v", err)
}
err = ioutil.WriteFile(wd+"/ro/subdir/file", []byte("hello"), 0644)
if err != nil {
t.Fatalf("WriteFile: %v", err)
}
setRecursiveWritable(t, wd+"/ro", false)
err = os.Chmod(wd+"/mnt/subdir/file", 0060)
if err != nil {
t.Fatalf("Chmod: %v", err)
}
fRo, err := os.Lstat(wd + "/ro/subdir")
if err != nil {
t.Fatalf("Lstat: %v", err)
}
fRw, err := os.Lstat(wd + "/rw/subdir")
if err != nil {
t.Fatalf("Lstat: %v", err)
}
// TODO - need to update timestamps after promoteDirsTo calls,
// not during.
if false && fRo.ModTime().Equal(fRw.ModTime()) {
t.Errorf("Changed timestamps on promoted subdir: ro %v rw %v", fRo.ModTime(), fRw.ModTime())
}
if fRo.Mode().Perm()|0200 != fRw.Mode().Perm() {
t.Errorf("Changed mode ro: %v, rw: %v", fRo.Mode(), fRw.Mode())
}
}
func TestUnionFsCheckHiddenFiles(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
err := ioutil.WriteFile(wd+"/ro/hidden", []byte("bla"), 0644)
if err != nil {
t.Fatalf("WriteFile: %v", err)
}
err = ioutil.WriteFile(wd+"/ro/not_hidden", []byte("bla"), 0644)
if err != nil {
t.Fatalf("WriteFile: %v", err)
}
setRecursiveWritable(t, wd+"/ro", false)
fi, _ := os.Lstat(wd + "/mnt/hidden")
if fi != nil {
t.Fatal("Lstat() should have failed", fi)
}
_, err = os.Lstat(wd + "/mnt/not_hidden")
if err != nil {
t.Fatalf("Lstat: %v", err)
}
names, err := Readdirnames(wd + "/mnt")
if err != nil {
t.Fatalf("Readdirnames: %v", err)
}
if len(names) != 1 || names[0] != "not_hidden" {
t.Fatal("unexpected names", names)
}
}
func TestUnionFSBarf(t *testing.T) {
wd, clean := setupUfs(t)
defer clean()
if err := os.Mkdir(wd+"/mnt/dir", 0755); err != nil {
t.Fatalf("os.Mkdir: %v", err)
}
if err := os.Mkdir(wd+"/mnt/dir2", 0755); err != nil {
t.Fatalf("os.Mkdir: %v", err)
}
if err := ioutil.WriteFile(wd+"/rw/dir/file", []byte("bla"), 0644); err != nil {
t.Fatalf("WriteFile: %v", err)
}
if _, err := os.Lstat(wd + "/mnt/dir/file"); err != nil {
t.Fatalf("Lstat: %v", err)
}
if err := os.Rename(wd+"/rw/dir/file", wd+"/rw/file"); err != nil {
t.Fatalf("os.Rename: %v", err)
}
if err := os.Rename(wd+"/mnt/file", wd+"/mnt/dir2/file"); err != nil {
t.Fatalf("os.Rename: %v", err)
}
}
unionfs/unionfs_xattr_test.go deleted 100644 → 0
View file @ 51ae026e
// Copyright 2016 the Go-FUSE Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package unionfs
import (
"os"
"sync/atomic"
"testing"
"time"
"github.com/hanwen/go-fuse/v2/fuse"
"github.com/hanwen/go-fuse/v2/fuse/nodefs"
"github.com/hanwen/go-fuse/v2/fuse/pathfs"
"github.com/hanwen/go-fuse/v2/internal/testutil"
)
type TestFS struct {
pathfs.FileSystem
xattrRead int64
}
func (fs *TestFS) GetAttr(path string, context *fuse.Context) (*fuse.Attr, fuse.Status) {
switch path {
case "":
return &fuse.Attr{Mode: fuse.S_IFDIR | 0755}, fuse.OK
case "file":
return &fuse.Attr{Mode: fuse.S_IFREG | 0755}, fuse.OK
}
return nil, fuse.ENOENT
}
func (fs *TestFS) GetXAttr(path string, name string, context *fuse.Context) ([]byte, fuse.Status) {
if path == "file" && name == "user.attr" {
atomic.AddInt64(&fs.xattrRead, 1)
return []byte{42}, fuse.OK
}
return nil, fuse.ENOATTR
}
func TestXAttrCaching(t *testing.T) {
wd := testutil.TempDir()
defer os.RemoveAll(wd)
os.Mkdir(wd+"/mnt", 0700)
err := os.Mkdir(wd+"/rw", 0700)
if err != nil {
t.Fatalf("Mkdir failed: %v", err)
}
rwFS := pathfs.NewLoopbackFileSystem(wd + "/rw")
roFS := &TestFS{
FileSystem: pathfs.NewDefaultFileSystem(),
}
ufs, err := NewUnionFs([]pathfs.FileSystem{rwFS,
NewCachingFileSystem(roFS, entryTTL)}, testOpts)
if err != nil {
t.Fatalf("NewUnionFs: %v", err)
}
opts := &nodefs.Options{
EntryTimeout: entryTTL / 2,
AttrTimeout: entryTTL / 2,
NegativeTimeout: entryTTL / 2,
Debug: testutil.VerboseTest(),
LookupKnownChildren: true,
}
pathfs := pathfs.NewPathNodeFs(ufs,
&pathfs.PathNodeFsOptions{ClientInodes: true,
Debug: testutil.VerboseTest()})
server, _, err := nodefs.MountRoot(wd+"/mnt", pathfs.Root(), opts)
if err != nil {
t.Fatalf("MountNodeFileSystem failed: %v", err)
}
defer server.Unmount()
go server.Serve()
server.WaitMount()
start := time.Now()
if fi, err := os.Lstat(wd + "/mnt"); err != nil || !fi.IsDir() {
t.Fatalf("root not readable: %v, %v", err, fi)
}
buf := make([]byte, 1024)
n, err := Getxattr(wd+"/mnt/file", "user.attr", buf)
if err != nil {
t.Fatalf("Getxattr: %v", err)
}
want := "\x2a"
got := string(buf[:n])
if got != want {
t.Fatalf("Got %q want %q", got, err)
}
time.Sleep(entryTTL / 3)
n, err = Getxattr(wd+"/mnt/file", "user.attr", buf)
if err != nil {
t.Fatalf("Getxattr: %v", err)
}
got = string(buf[:n])
if got != want {
t.Fatalf("Got %q want %q", got, err)
}
time.Sleep(entryTTL / 3)
// Make sure that an interceding Getxattr() to a filesystem that doesn't implement GetXAttr() doesn't affect future calls.
Getxattr(wd, "whatever", buf)
n, err = Getxattr(wd+"/mnt/file", "user.attr", buf)
if err != nil {
t.Fatalf("Getxattr: %v", err)
}
got = string(buf[:n])
if got != want {
t.Fatalf("Got %q want %q", got, err)
}
if time.Now().Sub(start) >= entryTTL {
// If we run really slowly, this test will spuriously
// fail.
t.Skip("test took too long.")
}
actual := atomic.LoadInt64(&roFS.xattrRead)
if actual != 1 {
t.Errorf("got xattrRead=%d, want 1", actual)
}
}
unionfs/unionfs_xattr_test_darwin.go deleted 100644 → 0
View file @ 51ae026e
// Copyright 2016 the Go-FUSE Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package unionfs
import (
"syscall"
"unsafe"
)
// Darwin doesn't have support for syscall.Getxattr() so we pull it into its own file and implement it by hand on Darwin.
func Getxattr(path string, attr string, dest []byte) (sz int, err error) {
var _p0 *byte
_p0, err = syscall.BytePtrFromString(path)
if err != nil {
return
}
var _p1 *byte
_p1, err = syscall.BytePtrFromString(attr)
if err != nil {
return
}
var _p2 unsafe.Pointer
if len(dest) > 0 {
_p2 = unsafe.Pointer(&dest[0])
} else {
var _zero uintptr
_p2 = unsafe.Pointer(&_zero)
}
r0, _, e1 := syscall.Syscall6(syscall.SYS_GETXATTR, uintptr(unsafe.Pointer(_p0)), uintptr(unsafe.Pointer(_p1)), uintptr(_p2), uintptr(len(dest)), 0, 0)
sz = int(r0)
if e1 != 0 {
err = e1
}
return
}
unionfs/unionfs_xattr_test_linux.go deleted 100644 → 0
View file @ 51ae026e
// Copyright 2016 the Go-FUSE Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package unionfs
import (
"syscall"
)
// Darwin doesn't have support for syscall.Getxattr() so we pull it into its own file and implement it by hand on Darwin.
func Getxattr(path string, attr string, dest []byte) (sz int, err error) {
return syscall.Getxattr(path, attr, dest)
}
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7