- 24 May, 2023 35 commits
-
-
David Howells authored
Fix bio_flagged() so that multiple instances of it, such as: if (bio_flagged(bio, BIO_PAGE_REFFED) || bio_flagged(bio, BIO_PAGE_PINNED)) can be combined by the gcc optimiser into a single test in assembly (arguably, this is a compiler optimisation issue[1]). The missed optimisation stems from bio_flagged() comparing the result of the bitwise-AND to zero. This results in an out-of-line bio_release_page() being compiled to something like: <+0>: mov 0x14(%rdi),%eax <+3>: test $0x1,%al <+5>: jne 0xffffffff816dac53 <bio_release_pages+11> <+7>: test $0x2,%al <+9>: je 0xffffffff816dac5c <bio_release_pages+20> <+11>: movzbl %sil,%esi <+15>: jmp 0xffffffff816daba1 <__bio_release_pages> <+20>: jmp 0xffffffff81d0b800 <__x86_return_thunk> However, the test is superfluous as the return type is bool. Removing it results in: <+0>: testb $0x3,0x14(%rdi) <+4>: je 0xffffffff816e4af4 <bio_release_pages+15> <+6>: movzbl %sil,%esi <+10>: jmp 0xffffffff816dab7c <__bio_release_pages> <+15>: jmp 0xffffffff81d0b7c0 <__x86_return_thunk> instead. Also, the MOVZBL instruction looks unnecessary[2] - I think it's just 're-booling' the mark_dirty parameter. Signed-off-by:
David Howells <dhowells@redhat.com> Reviewed-by:
Christoph Hellwig <hch@lst.de> Reviewed-by:
John Hubbard <jhubbard@nvidia.com> cc: Jens Axboe <axboe@kernel.dk> cc: linux-block@vger.kernel.org Link: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108370 [1] Link: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108371 [2] Link: https://lore.kernel.org/r/167391056756.2311931.356007731815807265.stgit@warthog.procyon.org.uk/ # v6 Reviewed-by:
Christian Brauner <brauner@kernel.org> Reviewed-by:
Jan Kara <jack@suse.cz> Link: https://lore.kernel.org/r/20230522205744.2825689-3-dhowells@redhat.comSigned-off-by:
Jens Axboe <axboe@kernel.dk>
-
David Howells authored
ZERO_PAGE can't go away, no need to hold an extra reference. Signed-off-by:
David Hildenbrand <david@redhat.com> Reviewed-by:
Dave Chinner <dchinner@redhat.com> Reviewed-by:
-
Jens Axboe authored
Merge splice bits as subsequent block cleanups and improvements for DIO depend on them. * for-6.5/splice: (31 commits) splice: kdoc for filemap_splice_read() and copy_splice_read() iov_iter: Kill ITER_PIPE splice: Remove generic_file_splice_read() splice: Use filemap_splice_read() instead of generic_file_splice_read() cifs: Use filemap_splice_read() trace: Convert trace/seq to use copy_splice_read() zonefs: Provide a splice-read wrapper xfs: Provide a splice-read wrapper orangefs: Provide a splice-read wrapper ocfs2: Provide a splice-read wrapper ntfs3: Provide a splice-read wrapper nfs: Provide a splice-read wrapper f2fs: Provide a splice-read wrapper ext4: Provide a splice-read wrapper ecryptfs: Provide a splice-read wrapper ceph: Provide a splice-read wrapper afs: Provide a splice-read wrapper 9p: Add splice_read wrapper net: Make sock_splice_read() use copy_splice_read() by default tty, proc, kernfs, random: Use copy_splice_read() ...
-
David Howells authored
Provide kerneldoc comments for filemap_splice_read() and copy_splice_read(). Signed-off-by:
-
David Howells authored
The ITER_PIPE-type iterator was only used by generic_file_splice_read() and that has been replaced and removed. This leaves ITER_PIPE unused - so remove it too. Signed-off-by:
-
David Howells authored
Remove generic_file_splice_read() as it has been replaced with calls to filemap_splice_read() and copy_splice_read(). With this, ITER_PIPE is no longer used. Signed-off-by:
-
David Howells authored
Replace pointers to generic_file_splice_read() with calls to filemap_splice_read(). Signed-off-by:
-
David Howells authored
Make cifs use filemap_splice_read() rather than doing its own version of generic_file_splice_read(). Signed-off-by:
Paulo Alcantara (SUSE) <pc@manguebit.com> cc: Jens Axboe <axboe@kernel.dk> cc: Steve French <smfrench@gmail.com> cc: Al Viro <viro@zeniv.linux.org.uk> cc: David Hildenbrand <david@redhat.com> cc: John Hubbard <jhubbard@nvidia.com> cc: linux-mm@kvack.org cc: linux-block@vger.kernel.org cc: linux-cifs@vger.kernel.org cc: linux-fsdevel@vger.kernel.org Link: https://lore.kernel.org/r/20230522135018.2742245-28-dhowells@redhat.comSigned-off-by:
-
David Howells authored
For the splice from the trace seq buffer, just use copy_splice_read(). In the future, something better can probably be done by gifting pages from seq->buf into the pipe, but that would require changing seq->buf into a vmap over an array of pages. Signed-off-by:
-
David Howells authored
Provide a splice_read wrapper for zonefs. This does some checks before proceeding and locks the inode across the call to filemap_splice_read() and a size check in case of truncation. Splicing from direct I/O is handled by the caller. Signed-off-by:
Damien Le Moal <dlemoal@kernel.org> Link: https://lore.kernel.org/r/20230522135018.2742245-26-dhowells@redhat.comSigned-off-by:
-
David Howells authored
Provide a splice_read wrapper for XFS. This does a stat count and a shutdown check before proceeding, then emits a new trace line and locks the inode across the call to filemap_splice_read() and adds to the stats afterwards. Splicing from direct I/O or DAX is handled by the caller. Signed-off-by:
-
David Howells authored
Provide a splice_read wrapper for ocfs2. This increments the read stats and then locks the inode across the call to filemap_splice_read() and a revalidation of the mapping. Splicing from direct I/O is done by the caller. Signed-off-by:
-
David Howells authored
Provide a splice_read wrapper for ocfs2. This emits trace lines and does an atime lock/update before calling filemap_splice_read(). Splicing from direct I/O is handled by the caller. A couple of new tracepoints are added for this purpose. Signed-off-by:
Joseph Qi <joseph.qi@linux.alibaba.com> cc: Christoph Hellwig <hch@lst.de> cc: Al Viro <viro@zeniv.linux.org.uk> cc: Jens Axboe <axboe@kernel.dk> cc: Mark Fasheh <mark@fasheh.com> cc: Joel Becker <jlbec@evilplan.org> cc: ocfs2-devel@oss.oracle.com cc: linux-fsdevel@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-mm@kvack.org Link: https://lore.kernel.org/r/20230522135018.2742245-23-dhowells@redhat.comSigned-off-by:
-
David Howells authored
Provide a splice_read wrapper for NTFS3 to perform various checks before allowing the operation to proceed. Signed-off-by:
-
David Howells authored
Provide a splice_read wrapper for NFS. This locks the inode around filemap_splice_read() and revalidates the mapping. Splicing from direct I/O is handled by the caller. Signed-off-by:
-
David Howells authored
Provide a splice_read wrapper for f2fs. This does some checks and tracing before calling filemap_splice_read() and will update the iostats afterwards. Direct I/O is handled by the caller. Signed-off-by:
-
David Howells authored
Provide a splice_read wrapper for Ext4. This does the inode shutdown check before proceeding. Splicing from DAX files and O_DIRECT fds is handled by the caller. Signed-off-by:
Theodore Ts'o <tytso@mit.edu> cc: Al Viro <viro@zeniv.linux.org.uk> cc: Jens Axboe <axboe@kernel.dk> cc: Andreas Dilger <adilger.kernel@dilger.ca> cc: linux-ext4@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-mm@kvack.org Link: https://lore.kernel.org/r/20230522135018.2742245-19-dhowells@redhat.comSigned-off-by:
-
David Howells authored
Provide a splice_read wrapper for ecryptfs to update the access time on the lower file after the operation. Splicing from a direct I/O fd will update the access time when ->read_iter() is called. Signed-off-by:
-
David Howells authored
Provide a splice_read wrapper for Ceph. This does the inode shutdown check before proceeding and jumps to copy_splice_read() if the file has inline data or is a synchronous file. We try and get FILE_RD and either FILE_CACHE and/or FILE_LAZYIO caps and hold them across filemap_splice_read(). If we fail to get FILE_CACHE or FILE_LAZYIO capabilities, we use copy_splice_read() instead. Signed-off-by:
Xiubo Li <xiubli@redhat.com> cc: Christoph Hellwig <hch@lst.de> cc: Al Viro <viro@zeniv.linux.org.uk> cc: Jens Axboe <axboe@kernel.dk> cc: Ilya Dryomov <idryomov@gmail.com> cc: Jeff Layton <jlayton@kernel.org> cc: ceph-devel@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-mm@kvack.org Link: https://lore.kernel.org/r/20230522135018.2742245-17-dhowells@redhat.comSigned-off-by:
-
David Howells authored
Provide a splice_read wrapper for AFS to call afs_validate() before going into generic_file_splice_read() so that we're likely to have a callback promise from the server. Signed-off-by:
-
David Howells authored
Add a splice_read wrapper for 9p. We should use copy_splice_read() if 9PL_DIRECT is set and filemap_splice_read() otherwise. Note that this doesn't seem to be particularly related to O_DIRECT. Signed-off-by:
-
David Howells authored
Make sock_splice_read() use copy_splice_read() by default as file_splice_read() will return immediately with 0 as a socket has no pagecache and is a zero-size file. Signed-off-by:
-
David Howells authored
Use copy_splice_read() for tty, procfs, kernfs and random files rather than going through generic_file_splice_read() as they just copy the file into the output buffer and don't splice pages. This avoids the need for them to have a ->read_folio() to satisfy filemap_splice_read(). Signed-off-by:
Greg Kroah-Hartman <gregkh@linuxfoundation.org> cc: Christoph Hellwig <hch@lst.de> cc: Jens Axboe <axboe@kernel.dk> cc: Al Viro <viro@zeniv.linux.org.uk> cc: John Hubbard <jhubbard@nvidia.com> cc: David Hildenbrand <david@redhat.com> cc: Matthew Wilcox <willy@infradead.org> cc: Miklos Szeredi <miklos@szeredi.hu> cc: Arnd Bergmann <arnd@arndb.de> cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org Link: https://lore.kernel.org/r/20230522135018.2742245-13-dhowells@redhat.comSigned-off-by:
-
David Howells authored
Implement splice-read for coda by passing the request down a layer rather than going through generic_file_splice_read() which is going to be changed to assume that ->read_folio() is present on buffered files. Signed-off-by:
Jan Harkes <jaharkes@cs.cmu.edu> cc: Christoph Hellwig <hch@lst.de> cc: Jens Axboe <axboe@kernel.dk> cc: Al Viro <viro@zeniv.linux.org.uk> cc: John Hubbard <jhubbard@nvidia.com> cc: David Hildenbrand <david@redhat.com> cc: Matthew Wilcox <willy@infradead.org> cc: coda@cs.cmu.edu cc: codalist@coda.cs.cmu.edu cc: linux-unionfs@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org Link: https://lore.kernel.org/r/20230522135018.2742245-12-dhowells@redhat.comSigned-off-by:
-
David Howells authored
Implement splice-read for overlayfs by passing the request down a layer rather than going through generic_file_splice_read() which is going to be changed to assume that ->read_folio() is present on buffered files. Signed-off-by:
-
David Howells authored
The new filemap_splice_read() has an implicit expectation via filemap_get_pages() that ->read_folio() exists if ->readahead() doesn't fully populate the pagecache of the file it is reading from[1], potentially leading to a jump to NULL if this doesn't exist. shmem, however, (and by extension, tmpfs, ramfs and rootfs), doesn't have ->read_folio(), Work around this by equipping shmem with its own splice-read implementation, based on filemap_splice_read(), but able to paste in zero_page when there's a page missing. Signed-off-by:
-
David Howells authored
Make a read splice from a DAX file go directly to copy_splice_read() to do the reading as filemap_splice_read() is unlikely to find any pagecache to splice. I think this affects only erofs, Ext2, Ext4, fuse and XFS. Signed-off-by:
Gao Xiang <hsiangkao@linux.alibaba.com> cc: Al Viro <viro@zeniv.linux.org.uk> cc: Jens Axboe <axboe@kernel.dk> cc: linux-erofs@lists.ozlabs.org cc: linux-ext4@vger.kernel.org cc: linux-xfs@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-mm@kvack.org Link: https://lore.kernel.org/r/20230522135018.2742245-9-dhowells@redhat.comSigned-off-by:
-
David Howells authored
Make a read splice from a file descriptor that's open O_DIRECT use copy_splice_read() to do the reading as filemap_splice_read() is unlikely to find any pagecache to splice. Signed-off-by:
-
David Howells authored
Make vfs_splice_read() return immediately if the length is 0. Signed-off-by:
-
David Howells authored
Rename do_splice_to() to vfs_splice_read() and export it so that it can be used as a helper when calling down to a lower layer filesystem as it performs all the necessary checks[1]. Signed-off-by:
-
David Howells authored
Do a couple of cleanups to copy_splice_read(): (1) Cast to struct page **, not void *. (2) Simplify the calculation of the number of pages to keep/reclaim in copy_splice_read(). Suggested-by:Christoph Hellwig <hch@infradead.org> Signed-off-by:
-
David Howells authored
Rename direct_splice_read() to copy_splice_read() to better reflect as to what it does. Suggested-by:
-
David Howells authored
Make filemap_splice_read() check s_maxbytes analogously to filemap_read(). Signed-off-by:
-
David Howells authored
Fix filemap_splice_read() to use file->f_mapping->host, not file->f_inode, as the source of the file size because in the case of a block device, file->f_inode points to the block-special file (which is typically 0 length) and not the backing store. Fixes: 07073eb0 ("splice: Add a func to do a splice from a buffered file without ITER_PIPE") Signed-off-by:
-
Hengqi Chen authored
Currently, several BCC ([0]) tools (biosnoop/biostacks/biotop) use kprobes to blk_account_io_start/blk_account_io_done to implement their functionalities. This is fragile because the target kernel functions may be renamed ([1]) or inlined ([2]). So introduce two new tracepoints for such use cases. [0]: https://github.com/iovisor/bcc [1]: https://github.com/iovisor/bcc/issues/3954 [2]: https://github.com/iovisor/bcc/issues/4261Tested-by:
Francis Laniel <flaniel@linux.microsoft.com> Signed-off-by:
Hengqi Chen <hengqi.chen@gmail.com> Tested-by:
Yonghong Song <yhs@fb.com> Link: https://lore.kernel.org/r/20230520084057.1467003-1-hengqi.chen@gmail.comSigned-off-by:
-
- 23 May, 2023 2 commits
-
-
Yu Kuai authored
commit 50e34d78 ("block: disable the elevator int del_gendisk") move rq_qos_exit() from disk_release() to del_gendisk(), this will introduce some problems: 1) If rq_qos_add() is triggered by enabling iocost/iolatency through cgroupfs, then it can concurrent with del_gendisk(), it's not safe to write 'q->rq_qos' concurrently. 2) Activate cgroup policy that is relied on rq_qos will call rq_qos_add() and blkcg_activate_policy(), and if rq_qos_exit() is called in the middle, null-ptr-dereference will be triggered in blkcg_activate_policy(). 3) blkg_conf_open_bdev() can call blkdev_get_no_open() first to find the disk, then if rq_qos_exit() from del_gendisk() is done before rq_qos_add(), then memory will be leaked. This patch add a new disk level mutex 'rq_qos_mutex': 1) The lock will protect rq_qos_exit() directly. 2) For wbt that doesn't relied on blk-cgroup, rq_qos_add() can only be called from disk initialization for now because wbt can't be destructed until rq_qos_exit(), so it's safe not to protect wbt for now. Hoever, in case that rq_qos dynamically destruction is supported in the furture, this patch also protect rq_qos_add() from wbt_init() directly, this is enough because blk-sysfs already synchronize writers with disk removal. 3) For iocost and iolatency, in order to synchronize disk removal and cgroup configuration, the lock is held after blkdev_get_no_open() from blkg_conf_open_bdev(), and is released in blkg_conf_exit(). In order to fix the above memory leak, disk_live() is checked after holding the new lock. Fixes: 50e34d78 ("block: disable the elevator int del_gendisk") Signed-off-by:
Yu Kuai <yukuai3@huawei.com> Acked-by:
Tejun Heo <tj@kernel.org> Link: https://lore.kernel.org/r/20230414084008.2085155-1-yukuai1@huaweicloud.comSigned-off-by:
-
Li Nan authored
op &= REQ_OP_MASK in blk_op_is_passthrough() is exactly what req_op() do. Therefore, it is redundant to call req_op() for blk_op_is_passthrough(). Signed-off-by:
Li Nan <linan122@huawei.com> Reviewed-by:
Chaitanya Kulkarni <kch@nvidia.com> Link: https://lore.kernel.org/r/20230522085355.1740772-1-linan666@huaweicloud.comSigned-off-by:
-
- 21 May, 2023 1 commit
-
-
Ming Lei authored
Return type of iov_iter_get_pages2() is ssize_t instead of size_t, so fix it. Fixes: 981f95a5 ("ublk: cleanup ublk_copy_user_pages") Reported-by:
kernel test robot <lkp@intel.com> Reported-by:
Julia Lawall <julia.lawall@inria.fr> Signed-off-by:
Ming Lei <ming.lei@redhat.com> Link: https://lore.kernel.org/r/20230520151134.459679-1-ming.lei@redhat.comSigned-off-by:
-
- 20 May, 2023 2 commits
-
-
Christoph Hellwig authored
For direct I/O writes that issues more than a single bio, the plugging is already done in __blkdev_direct_IO. For synchronous buffered writes the plugging is done deep down in writeback_inodes_wb / wb_writeback. For the other cases there is no point in plugging as as single bio or no bio at all is submitted. Signed-off-by:
-
Bart Van Assche authored
Check bfqq->dispatched for each BFQ queue instead of checking it for an invalid bfqq pointer. Fixes: 3e49c1e4 ("block: BFQ: Add several invariant checks") Signed-off-by:
Bart Van Assche <bvanassche@acm.org> Link: https://lore.kernel.org/r/20230519220347.3643295-1-bvanassche@acm.orgSigned-off-by:
-
