- 17 May, 2022 28 commits
-
-
Xin Yin authored
Use asynchronous io to read data from fscache may greatly improve IO bandwidth for sequential buffered read scenario. Change erofs_fscache_read_folios to erofs_fscache_read_folios_async, and read data from fscache asynchronously. Make .readpage()/.readahead() to use this new helper. Signed-off-by:
Xin Yin <yinxin.x@bytedance.com> Reviewed-by:
Jeffle Xu <jefflexu@linux.alibaba.com> Signed-off-by:
Chao Yu <chao@kernel.org> [ Gao Xiang: minor styling changes. ] Signed-off-by:
Gao Xiang <hsiangkao@linux.alibaba.com>
-
Jeffle Xu authored
Introduce 'fsid' mount option to enable on-demand read sementics, in which case, erofs will be mounted from data blobs. Users could specify the name of primary data blob by this mount option. Signed-off-by:
Zichen Tian <tianzichen@kuaishou.com> Tested-by:
Jia Zhu <zhujia.zj@bytedance.com> Tested-by:
Yan Song <yansong.ys@antgroup.com> Signed-off-by:
-
Jeffle Xu authored
Implement fscache-based data readahead. Also registers an individual bdi for each erofs instance to enable readahead. Signed-off-by:
-
Jeffle Xu authored
Implement the data plane of reading data from data blobs over fscache for inline layout. For the heading non-inline part, the data plane for non-inline layout is reused, while only the tail packing part needs special handling. Signed-off-by:
-
Jeffle Xu authored
Implement the data plane of reading data from data blobs over fscache for non-inline layout. Signed-off-by:
-
Jeffle Xu authored
Implement the data plane of reading metadata from primary data blob over fscache. Signed-off-by:
-
Jeffle Xu authored
Similar to the multi-device mode, erofs could be mounted from one primary data blob (mandatory) and multiple extra data blobs (optional). Register fscache context for each extra data blob. Signed-off-by:
-
Jeffle Xu authored
Registers fscache context for primary data blob. Also move the initialization of s_op and related fields forward, since anonymous inode will be allocated under the super block when registering the fscache context. Something worth mentioning about the cleanup routine. 1. The fscache context will instantiate anonymous inodes under the super block. Release these anonymous inodes when .put_super() is called, or we'll get "VFS: Busy inodes after unmount." warning. 2. The fscache context is initialized prior to the root inode. If .kill_sb() is called when mount failed, .put_super() won't be called when root inode has not been initialized yet. Thus .kill_sb() shall also contain the cleanup routine. Signed-off-by:
-
Jeffle Xu authored
Add erofs_fscache_read_folios() helper reading from fscache. It supports on-demand read semantics. That is, it will make the backend prepare for the data when cache miss. Once data ready, it will read from the cache. This helper can then be used to implement .readpage()/.readahead() of on-demand read semantics. Signed-off-by:
-
Jeffle Xu authored
Introduce one anonymous inode for data blobs so that erofs can cache metadata directly within such anonymous inode. Signed-off-by:
-
Jeffle Xu authored
Introduce a context structure for managing data blobs, and helper functions for initializing and cleaning up this context structure. Signed-off-by:
-
Jeffle Xu authored
A new fscache based mode is going to be introduced for erofs, in which case on-demand read semantics is implemented through fscache. As the first step, register fscache volume for each erofs filesystem. That means, data blobs can not be shared among erofs filesystems. In the following iteration, we are going to introduce the domain semantics, in which case several erofs filesystems can belong to one domain, and data blobs can be shared among these erofs filesystems of one domain. Signed-off-by:
-
Jeffle Xu authored
Until then erofs is exactly blockdev based filesystem. A new fscache-based mode is going to be introduced for erofs to support scenarios where on-demand read semantics is needed, e.g. container image distribution. In this case, erofs could be mounted from data blobs through fscache. Add a helper checking which mode erofs works in, and twist the code in preparation for the upcoming fscache mode. Signed-off-by:
-
Jeffle Xu authored
... so that it can be used in the following introduced fscache mode. Signed-off-by:
-
Jeffle Xu authored
Document new user interface introduced by on-demand read mode. Signed-off-by:
David Howells <dhowells@redhat.com> Signed-off-by:
-
Jeffle Xu authored
Add tracepoints for on-demand read mode. Currently following tracepoints are added: OPEN request / COPEN reply CLOSE request READ request / CREAD reply write through anonymous fd release of anonymous fd Signed-off-by:
-
Jeffle Xu authored
Enable on-demand read mode by adding an optional parameter to the "bind" command. On-demand mode will be turned on when this parameter is "ondemand", i.e. "bind ondemand". Otherwise cachefiles will work in the original mode. Signed-off-by:
-
Jeffle Xu authored
Implement the data plane of on-demand read mode. The early implementation [1] place the entry to cachefiles_ondemand_read() in fscache_read(). However, fscache_read() can only detect if the requested file range is fully cache miss, whilst we need to notify the user daemon as long as there's a hole inside the requested file range. Thus the entry is now placed in cachefiles_prepare_read(). When working in on-demand read mode, once a hole detected, the read routine will send a READ request to the user daemon. The user daemon needs to fetch the data and write it to the cache file. After sending the READ request, the read routine will hang there, until the READ request is handled by the user daemon. Then it will retry to read from the same file range. If no progress encountered, the read routine will fail then. A new NETFS_SREQ_ONDEMAND flag is introduced to indicate that on-demand read should be done when a cache miss encountered. [1] https://lore.kernel.org/all/20220406075612.60298-6-jefflexu@linux.alibaba.com/ #v8 Signed-off-by:
-
Jeffle Xu authored
Notify the user daemon that cookie is going to be withdrawn, providing a hint that the associated anonymous fd can be closed. Be noted that this is only a hint. The user daemon may close the associated anonymous fd when receiving the CLOSE request, then it will receive another anonymous fd when the cookie gets looked up. Or it may ignore the CLOSE request, and keep writing data through the anonymous fd. However the next time the cookie gets looked up, the user daemon will still receive another new anonymous fd. Signed-off-by:
-
Jeffle Xu authored
Add a refcount to avoid the deadlock in on-demand read mode. The on-demand read mode will pin the corresponding cachefiles object for each anonymous fd. The cachefiles object is unpinned when the anonymous fd gets closed. When the user daemon exits and the fd of "/dev/cachefiles" device node gets closed, it will wait for all cahcefiles objects getting withdrawn. Then if there's any anonymous fd getting closed after the fd of the device node, the user daemon will hang forever, waiting for all objects getting withdrawn. To fix this, add a refcount indicating if there's any object pinned by anonymous fds. The cachefiles cache gets unbound and withdrawn when the refcount is decreased to 0. It won't change the behaviour of the original mode, in which case the cachefiles cache gets unbound and withdrawn as long as the fd of the device node gets closed. Signed-off-by:
-
Jeffle Xu authored
Fscache/CacheFiles used to serve as a local cache for a remote networking fs. A new on-demand read mode will be introduced for CacheFiles, which can boost the scenario where on-demand read semantics are needed, e.g. container image distribution. The essential difference between these two modes is seen when a cache miss occurs: In the original mode, the netfs will fetch the data from the remote server and then write it to the cache file; in on-demand read mode, fetching the data and writing it into the cache is delegated to a user daemon. As the first step, notify the user daemon when looking up cookie. In this case, an anonymous fd is sent to the user daemon, through which the user daemon can write the fetched data to the cache file. Since the user daemon may move the anonymous fd around, e.g. through dup(), an object ID uniquely identifying the cache file is also attached. Also add one advisory flag (FSCACHE_ADV_WANT_CACHE_SIZE) suggesting that the cache file size shall be retrieved at runtime. This helps the scenario where one cache file contains multiple netfs files, e.g. for the purpose of deduplication. In this case, netfs itself has no idea the size of the cache file, whilst the user daemon should give the hint on it. Signed-off-by:
-
Jeffle Xu authored
Extract the generic routine of writing data to cache files, and make it generally available. This will be used by the following patch implementing on-demand read mode. Since it's called inside CacheFiles module, make the interface generic and unrelated to netfs_cache_resources. It is worth noting that, ki->inval_counter is not initialized after this cleanup. It shall not make any visible difference, since inval_counter is no longer used in the write completion routine, i.e. cachefiles_write_complete(). Signed-off-by:
-
Chao Yu authored
This patch enables idmapped mounts for erofs, since all dedicated helpers for this functionality existsm, so, in this patch we just pass down the user_namespace argument from the VFS methods to the relevant helpers. Simple idmap example on erofs image: 1. mkdir dir 2. touch dir/file 3. mkfs.erofs erofs.img dir 4. mount -t erofs -o loop erofs.img /mnt/erofs/ 5. ls -ln /mnt/erofs/ total 0 -rw-rw-r-- 1 1000 1000 0 May 17 15:26 file 6. mount-idmapped --map-mount b:1000:1001:1 /mnt/erofs/ /mnt/scratch_erofs/ 7. ls -ln /mnt/scratch_erofs/ total 0 -rw-rw-r-- 1 1001 1001 0 May 17 15:26 file Reviewed-by:
Christian Brauner (Microsoft) <brauner@kernel.org> Reviewed-by:
Chao Yu <chao.yu@oppo.com> Link: https://lore.kernel.org/r/20220517104103.3570721-1-chao@kernel.orgSigned-off-by:
-
Hongnan Li authored
Implement export operations in order to make EROFS support accessing inodes with filehandles so that it can be exported via NFS and used by overlayfs. Without this patch, 'exportfs -rv' will report: exportfs: /root/erofs_mp does not support NFS export Also tested with unionmount-testsuite and the testcase below passes now: ./run --ov --erofs --verify hard-link For more details about the testcase, see: https://github.com/amir73il/unionmount-testsuite/pull/6Signed-off-by:
Hongnan Li <hongnan.li@linux.alibaba.com> Reviewed-by:
-
Gao Xiang authored
I got some KASAN report as below: [ 46.959738] ================================================================== [ 46.960430] BUG: KASAN: use-after-free in z_erofs_shifted_transform+0x2bd/0x370 [ 46.960430] Read of size 4074 at addr ffff8880300c2f8e by task fssum/188 ... [ 46.960430] Call Trace: [ 46.960430] <TASK> [ 46.960430] dump_stack_lvl+0x41/0x5e [ 46.960430] print_report.cold+0xb2/0x6b7 [ 46.960430] ? z_erofs_shifted_transform+0x2bd/0x370 [ 46.960430] kasan_report+0x8a/0x140 [ 46.960430] ? z_erofs_shifted_transform+0x2bd/0x370 [ 46.960430] kasan_check_range+0x14d/0x1d0 [ 46.960430] memcpy+0x20/0x60 [ 46.960430] z_erofs_shifted_transform+0x2bd/0x370 [ 46.960430] z_erofs_decompress_pcluster+0xaae/0x1080 The root cause is that the tail pcluster won't be a complete filesystem block anymore. So if ztailpacking is used, the second part of an uncompressed tail pcluster may not be ``rq->pageofs_out``. Fixes: ab749bad ("erofs: support unaligned data decompression") Fixes: cecf864d ("erofs: support inline data decompression") Reviewed-by:
Yue Hu <huyue2@coolpad.com> Reviewed-by:
-
Gao Xiang authored
Fix some outdated comments and typos, hopefully helpful. Link: https://lore.kernel.org/r/20220506194612.117120-3-hsiangkao@linux.alibaba.comReviewed-by:
-
Gao Xiang authored
Some comments haven't been useful anymore since the code updated. Let's drop them instead. Link: https://lore.kernel.org/r/20220506194612.117120-2-hsiangkao@linux.alibaba.comReviewed-by:
-
Yue Hu authored
The big pcluster feature has been merged for a year, it has been mostly stable now. Signed-off-by:
-
- 24 Apr, 2022 8 commits
-
-
Linus Torvalds authored
-
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tipLinus Torvalds authored
Pull scheduler fix from Borislav Petkov: - Fix a corner case when calculating sched runqueue variables That fix also removes a check for a zero divisor in the code, without mentioning it. Vincent clarified that it's ok after I whined about it: https://lore.kernel.org/all/CAKfTPtD2QEyZ6ADd5WrwETMOX0XOwJGnVddt7VHgfURdqgOS-Q@mail.gmail.com/ * tag 'sched_urgent_for_v5.18_rc4' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: sched/pelt: Fix attach_entity_load_avg() corner case
-
git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linuxLinus Torvalds authored
Pull powerpc fixes from Michael Ellerman: - Partly revert a change to our timer_interrupt() that caused lockups with high res timers disabled. - Fix a bug in KVM TCE handling that could corrupt kernel memory. - Two commits fixing Power9/Power10 perf alternative event selection. Thanks to Alexey Kardashevskiy, Athira Rajeev, David Gibson, Frederic Barrat, Madhavan Srinivasan, Miguel Ojeda, and Nicholas Piggin. * tag 'powerpc-5.18-3' of git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux: powerpc/perf: Fix 32bit compile powerpc/perf: Fix power10 event alternatives powerpc/perf: Fix power9 event alternatives KVM: PPC: Fix TCE handling for VFIO powerpc/time: Always set decrementer in timer_interrupt()
-
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tipLinus Torvalds authored
Pull perf fixes from Borislav Petkov: - Add Sapphire Rapids CPU support - Fix a perf vmalloc-ed buffer mapping error (PERF_USE_VMALLOC in use) * tag 'perf_urgent_for_v5.18_rc4' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: perf/x86/cstate: Add SAPPHIRERAPIDS_X CPU support perf/core: Fix perf_mmap fail when CONFIG_PERF_USE_VMALLOC enabled
-
git://git.kernel.org/pub/scm/linux/kernel/git/ras/rasLinus Torvalds authored
Pull EDAC fix from Borislav Petkov: - Read the reported error count from the proper register on synopsys_edac * tag 'edac_urgent_for_v5.18_rc4' of git://git.kernel.org/pub/scm/linux/kernel/git/ras/ras: EDAC/synopsys: Read the error count from the correct register
-
Linus Torvalds authored
Since commit 559089e0 ("vmalloc: replace VM_NO_HUGE_VMAP with VM_ALLOW_HUGE_VMAP"), the use of hugepage mappings for vmalloc is an opt-in strategy, because it caused a number of problems that weren't noticed until x86 enabled it too. One of the issues was fixed by Nick Piggin in commit 3b8000ae ("mm/vmalloc: huge vmalloc backing pages should be split rather than compound"), but I'm still worried about page protection issues, and VM_FLUSH_RESET_PERMS in particular. However, like the hash table allocation case (commit f2edd118: "page_alloc: use vmalloc_huge for large system hash"), the use of kvmalloc() should be safe from any such games, since the returned pointer might be a SLUB allocation, and as such no user should reasonably be using it in any odd ways. We also know that the allocations are fairly large, since it falls back to the vmalloc case only when a kmalloc() fails. So using a hugepage mapping seems both safe and relevant. This patch does show a weakness in the opt-in strategy: since the opt-in flag is in the 'vm_flags', not the usual gfp_t allocation flags, very few of the usual interfaces actually expose it. That's not much of an issue in this case that already used one of the fairly specialized low-level vmalloc interfaces for the allocation, but for a lot of other vmalloc() users that might want to opt in, it's going to be very inconvenient. We'll either have to fix any compatibility problems, or expose it in the gfp flags (__GFP_COMP would have made a lot of sense) to allow normal vmalloc() users to use hugepage mappings. That said, the cases that really matter were probably already taken care of by the hash tabel allocation. Link: https://lore.kernel.org/all/20220415164413.2727220-1-song@kernel.org/ Link: https://lore.kernel.org/all/CAHk-=whao=iosX1s5Z4SF-ZGa-ebAukJoAdUJFk5SPwnofV+Vg@mail.gmail.com/ Cc: Nicholas Piggin <npiggin@gmail.com> Cc: Paul Menzel <pmenzel@molgen.mpg.de> Cc: Song Liu <songliubraving@fb.com> Cc: Rick Edgecombe <rick.p.edgecombe@intel.com> Cc: Andrew Morton <akpm@linux-foundation.org> Signed-off-by:
Linus Torvalds <torvalds@linux-foundation.org>
-
Song Liu authored
Use vmalloc_huge() in alloc_large_system_hash() so that large system hash (>= PMD_SIZE) could benefit from huge pages. Note that vmalloc_huge only allocates huge pages for systems with HAVE_ARCH_HUGE_VMALLOC. Signed-off-by:
Song Liu <song@kernel.org> Reviewed-by:
Christoph Hellwig <hch@lst.de> Reviewed-by:
Rik van Riel <riel@surriel.com> Signed-off-by:
-
git://git.samba.org/ksmbdLinus Torvalds authored
Pull ksmbd server fixes from Steve French: - cap maximum sector size reported to avoid mount problems - reference count fix - fix filename rename race * tag '5.18-rc3-ksmbd-fixes' of git://git.samba.org/ksmbd: ksmbd: set fixed sector size to FS_SECTOR_SIZE_INFORMATION ksmbd: increment reference count of parent fp ksmbd: remove filename in ksmbd_file
-
- 23 Apr, 2022 4 commits
-
-
git://git.kernel.org/pub/scm/linux/kernel/git/vgupta/arcLinus Torvalds authored
Pull ARC fixes from Vineet Gupta: - Assorted fixes * tag 'arc-5.18-rc4' of git://git.kernel.org/pub/scm/linux/kernel/git/vgupta/arc: ARC: remove redundant READ_ONCE() in cmpxchg loop ARC: atomic: cleanup atomic-llsc definitions arc: drop definitions of pgd_index() and pgd_offset{, _k}() entirely ARC: dts: align SPI NOR node name with dtschema ARC: Remove a redundant memset() ARC: fix typos in comments ARC: entry: fix syscall_trace_exit argument
-
git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsiLinus Torvalds authored
Pull SCSI fix from James Bottomley: "One fix for an information leak caused by copying a buffer to userspace without checking for error first in the sr driver" * tag 'scsi-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi: scsi: sr: Do not leak information in ioctl
-
git://git.kernel.org/pub/scm/linux/kernel/git/xen/tipLinus Torvalds authored
Pull xen fixes from Juergen Gross: "A simple cleanup patch and a refcount fix for Xen on Arm" * tag 'for-linus-5.18-rc4-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip: arm/xen: Fix some refcount leaks xen: Convert kmap() to kmap_local_page()
-
git://anongit.freedesktop.org/drm/drmLinus Torvalds authored
Pull more drm fixes from Dave Airlie: "Maarten was away, so Maxine stepped up and sent me the drm-fixes merge, so no point leaving it for another week. The big change is an OF revert around bridge/panels, it may have some driver fallout, but hopefully this revert gets them shook out in the next week easier. Otherwise it's a bunch of locking/refcounts across drivers, a radeon dma_resv logic fix and some raspberry pi panel fixes. panel: - revert of patch that broke panel/bridge issues dma-buf: - remove unused header file. amdgpu: - partial revert of locking change radeon: - fix dma_resv logic inversion panel: - pi touchscreen panel init fixes vc4: - build fix - runtime pm refcount fix vmwgfx: - refcounting fix" * tag 'drm-fixes-2022-04-23' of git://anongit.freedesktop.org/drm/drm: drm/amdgpu: partial revert "remove ctx->lock" v2 Revert "drm: of: Lookup if child node has panel or bridge" Revert "drm: of: Properly try all possible cases for bridge/panel detection" drm/vc4: Use pm_runtime_resume_and_get to fix pm_runtime_get_sync() usage drm/vmwgfx: Fix gem refcounting and memory evictions drm/vc4: Fix build error when CONFIG_DRM_VC4=y && CONFIG_RASPBERRYPI_FIRMWARE=m drm/panel/raspberrypi-touchscreen: Initialise the bridge in prepare drm/panel/raspberrypi-touchscreen: Avoid NULL deref if not initialised dma-buf-map: remove renamed header file drm/radeon: fix logic inversion in radeon_sync_resv
-
