- 18 Oct, 2021 40 commits
-
-
Christoph Hellwig authored
Use the sb_bdev_nr_blocks helper instead of open coding it. Signed-off-by:
Christoph Hellwig <hch@lst.de> Reviewed-by:
Kees Cook <keescook@chromium.org> Reviewed-by:
Jan Kara <jack@suse.cz> Link: https://lore.kernel.org/r/20211018101130.1838532-31-hch@lst.deSigned-off-by:
Jens Axboe <axboe@kernel.dk>
-
Christoph Hellwig authored
Use the sb_bdev_nr_blocks helper instead of open coding it. Signed-off-by:
-
Christoph Hellwig authored
Use the sb_bdev_nr_blocks helper instead of open coding it and clean up ntfs_fill_super a bit by moving an assignment a little earlier that has no negative side effects. Signed-off-by:
Anton Altaparmakov <anton@tuxera.com> Link: https://lore.kernel.org/r/20211018101130.1838532-29-hch@lst.deSigned-off-by:
-
Christoph Hellwig authored
Use the sb_bdev_nr_blocks helper instead of open coding it. Signed-off-by:
Dave Kleikamp <dave.kleikamp@oracle.com> Link: https://lore.kernel.org/r/20211018101130.1838532-28-hch@lst.deSigned-off-by:
-
Christoph Hellwig authored
Use the sb_bdev_nr_blocks helper instead of open coding it. Signed-off-by:
Chaitanya Kulkarni <kch@nvidia.com> Acked-by:
Theodore Ts'o <tytso@mit.edu> Link: https://lore.kernel.org/r/20211018101130.1838532-27-hch@lst.deSigned-off-by:
-
Christoph Hellwig authored
Add a helper to return the size of sb->s_bdev in sb->s_blocksize_bits based unites. Signed-off-by:
-
Christoph Hellwig authored
Use the proper helper to read the block device size. Signed-off-by:
-
Christoph Hellwig authored
Use the proper helper to read the block device size. Signed-off-by:
Phillip Lougher <phillip@squashfs.org.uk> Link: https://lore.kernel.org/r/20211018101130.1838532-24-hch@lst.deSigned-off-by:
-
Christoph Hellwig authored
Use the proper helper to read the block device size and remove two cargo culted checks that can't be false. Signed-off-by:
-
Christoph Hellwig authored
Use the proper helper to read the block device size. Signed-off-by:
-
Christoph Hellwig authored
Use the proper helper to read the block device size. Signed-off-by:
-
Christoph Hellwig authored
Use the proper helper to read the block device size. Signed-off-by:
Ryusuke Konishi <konishi.ryusuke@gmail.com> Link: https://lore.kernel.org/r/20211018101130.1838532-20-hch@lst.deSigned-off-by:
-
Christoph Hellwig authored
Use the proper helper to read the block device size. Signed-off-by:
-
Christoph Hellwig authored
Use the proper helper to read the block device size. Signed-off-by:
-
Christoph Hellwig authored
Use the proper helper to read the block device size. Signed-off-by:
-
Christoph Hellwig authored
Use the proper helper to read the block device size. Signed-off-by:
-
Christoph Hellwig authored
Use the proper helper to read the block device size. Signed-off-by:
-
Christoph Hellwig authored
Use the proper helper to read the block device size. Signed-off-by:
-
Christoph Hellwig authored
Use the proper helper to read the block device size. Signed-off-by:
Anand Jain <anand.jain@oracle.com> Reviewed-by:
David Sterba <dsterba@suse.com> Link: https://lore.kernel.org/r/20211018101130.1838532-13-hch@lst.deSigned-off-by:
-
Christoph Hellwig authored
Use the proper helper to read the block device size. Signed-off-by:
-
Christoph Hellwig authored
No need to convert from bdev to inode and back. Signed-off-by:
-
Christoph Hellwig authored
Use the proper helper to read the block device size. Signed-off-by:
-
Christoph Hellwig authored
Use the proper helper to read the block device size. Signed-off-by:
-
Christoph Hellwig authored
Use the proper helper to read the block device size. Signed-off-by:
-
Christoph Hellwig authored
Use the proper helper to read the block device size. Signed-off-by:
Song Liu <song@kernel.org> Link: https://lore.kernel.org/r/20211018101130.1838532-7-hch@lst.deSigned-off-by:
-
Christoph Hellwig authored
Use the proper helpers to read the block device size. Signed-off-by:
Mike Snitzer <snitzer@redhat.com> Link: https://lore.kernel.org/r/20211018101130.1838532-6-hch@lst.deSigned-off-by:
-
Christoph Hellwig authored
Use the proper helper to read the block device size. Signed-off-by:
Lee Duncan <lduncan@suse.com> Reviewed-by:
-
Christoph Hellwig authored
Use the equivalent block layer helper instead. Signed-off-by:
Coly Li <colyli@suse.de> Link: https://lore.kernel.org/r/20211018101130.1838532-4-hch@lst.deSigned-off-by:
-
Christoph Hellwig authored
Add a helper to query the size of a block device in bytes. This will be used to remove open coded access to ->bd_inode. Signed-off-by:
-
Christoph Hellwig authored
Ensure these are always available for inlines in the various block layer headers. Signed-off-by:
-
Jens Axboe authored
Trivial to do now, just need our own io_comp_batch on the stack and pass that in to the usual command completion handling. I pondered making this dependent on how many entries we had to process, but even for a single entry there's no discernable difference in performance or latency. Running a sync workload over io_uring: t/io_uring -b512 -d1 -s1 -c1 -p0 -F1 -B1 -n2 /dev/nvme1n1 /dev/nvme2n1 yields the below performance before the patch: IOPS=254820, BW=124MiB/s, IOS/call=1/1, inflight=(1 1) IOPS=251174, BW=122MiB/s, IOS/call=1/1, inflight=(1 1) IOPS=250806, BW=122MiB/s, IOS/call=1/1, inflight=(1 1) and the following after: IOPS=255972, BW=124MiB/s, IOS/call=1/1, inflight=(1 1) IOPS=251920, BW=123MiB/s, IOS/call=1/1, inflight=(1 1) IOPS=251794, BW=122MiB/s, IOS/call=1/1, inflight=(1 1) which definitely isn't slower, about the same if you factor in a bit of variance. For peak performance workloads, benchmarking shows a 2% improvement. Reviewed-by:
-
Jens Axboe authored
Wire up using an io_comp_batch for f_op->iopoll(). If the lower stack supports it, we can handle high rates of polled IO more efficiently. This raises the single core efficiency on my system from ~6.1M IOPS to ~6.6M IOPS running a random read workload at depth 128 on two gen2 Optane drives. Reviewed-by:
-
Jens Axboe authored
Take advantage of struct io_comp_batch, if passed in to the nvme poll handler. If it's set, rather than complete each request individually inline, store them in the io_comp_batch list. We only do so for requests that will complete successfully, anything else will be completed inline as before. Reviewed-by:
-
Jens Axboe authored
Instead of calling blk_mq_end_request() on a single request, add a helper that takes the new struct io_comp_batch and completes any request stored in there. Reviewed-by:
-
Jens Axboe authored
sbitmap currently only supports clearing tags one-by-one, add a helper that allows the caller to pass in an array of tags to clear. Signed-off-by: -
Jens Axboe authored
struct io_comp_batch contains a list head and a completion handler, which will allow completions to more effciently completed batches of IO. For now, no functional changes in this patch, we just define the io_comp_batch structure and add the argument to the file_operations iopoll handler. Reviewed-by:
-
Jens Axboe authored
Instead of open-coding the list additions, traversal, and removal, provide a basic set of helpers. Suggested-by:
Christoph Hellwig <hch@infradead.org> Signed-off-by:
-
Jens Axboe authored
Just like the blk_mq_ctx counterparts, we've got a bunch of counters in here that are only for debugfs and are of questionnable value. They are: - dispatched, index of how many requests were dispatched in one go - poll_{considered,invoked,success}, which track poll sucess rates. We're confident in the iopoll implementation at this point, don't bother tracking these. As a bonus, this shrinks each hardware queue from 576 bytes to 512 bytes, dropping a whole cacheline. Reviewed-by: -
Jens Axboe authored
These were added as part of early days debugging for blk-mq, and they are not really useful anymore. Rather than spend cycles updating them, just get rid of them. As a bonus, this shrinks the per-cpu software queue size from 256b to 192b. That's a whole cacheline less. Reviewed-by:
-
Pavel Begunkov authored
Add a local variable for rq_flags, it helps to compile out some of rq_flags reloads. Signed-off-by:
Pavel Begunkov <asml.silence@gmail.com> Signed-off-by:
-
