- 13 Apr, 2023 30 commits
-
-
Christoph Hellwig authored
Only blk_mq_run_hw_queue can call __blk_mq_delay_run_hw_queue with async=false, so move the handling there. With this __blk_mq_delay_run_hw_queue can be merged into blk_mq_delay_run_hw_queue. Signed-off-by:
Christoph Hellwig <hch@lst.de> Reviewed-by:
Damien Le Moal <dlemoal@kernel.org> Link: https://lore.kernel.org/r/20230413060651.694656-5-hch@lst.deSigned-off-by:
Jens Axboe <axboe@kernel.dk>
-
Christoph Hellwig authored
For the in-context dispatch, blk_mq_hctx_stopped is alredy checked in blk_mq_sched_dispatch_requests under blk_mq_run_dispatch_ops() protection. For the async dispatch case having a check before scheduling the work still makes sense to avoid needless workqueue scheduling, so just keep it for that case. Signed-off-by:
-
Christoph Hellwig authored
blk_mq_hctx_stopped is already checked in blk_mq_sched_dispatch_requests under blk_mq_run_dispatch_ops() protection, so remove the duplicate check. Signed-off-by:
-
Christoph Hellwig authored
__blk_mq_sched_dispatch_requests currently has duplicated logic for the cases where requests are on the hctx dispatch list or not. Merge the two with a new need_dispatch variable and remove a few pointless local variables. Signed-off-by:
-
Christoph Hellwig authored
Replace the boolean at_head argument with the same flags that are already passed to blk_mq_insert_request. Signed-off-by:
-
Christoph Hellwig authored
Instead of passing a bool at_head, pass down the full flags from the blk_mq_insert_request interface. Signed-off-by:
Bart Van Assche <bvanassche@acm.org> Reviewed-by:
-
Christoph Hellwig authored
Replace the boolean at_head argument with the same flags that are already passed to blk_mq_insert_request. Signed-off-by:
-
Christoph Hellwig authored
Replace the at_head bool with a flags argument that so far only contains a single BLK_MQ_INSERT_AT_HEAD value. This makes it much easier to grep for head insertions into the blk-mq dispatch queues. Signed-off-by:
-
Christoph Hellwig authored
blk_mq_add_to_requeue_list takes a bool parameter to control how to kick the requeue list at the end of the function. Move the call to blk_mq_kick_requeue_list to the callers that want it instead. Signed-off-by:
-
Christoph Hellwig authored
blk_mq_request_bypass_insert takes a bool parameter to control how to run the queue at the end of the function. Move the blk_mq_run_hw_queue call to the callers that want it instead. Signed-off-by:
-
Christoph Hellwig authored
blk_mq_insert_request takes two bool parameters to control how to run the queue at the end of the function. Move the blk_mq_run_hw_queue call to the callers that want it instead. Signed-off-by:
-
Christoph Hellwig authored
Due to the wildly different behavior based on the bypass_insert argument, not a whole lot of code in __blk_mq_try_issue_directly is actually shared between blk_mq_try_issue_directly and blk_mq_request_issue_directly. Remove __blk_mq_try_issue_directly and fold the code into the two callers instead. Signed-off-by:
-
Christoph Hellwig authored
Factor out a helper from __blk_mq_try_issue_directly in preparation of folding that function into its two callers. Signed-off-by:
-
Christoph Hellwig authored
Split the RQF_DONTPREP and RQF_SOFTBARRIER in separate branches to make the code more readable. Signed-off-by:
-
Christoph Hellwig authored
While both passthrough and flush requests call directly into blk_mq_request_bypass_insert, the parameters aren't the same. Split the handling into two separate conditionals and turn the whole function into an if/elif/elif/else flow instead of the gotos. Signed-off-by:
-
Christoph Hellwig authored
Just call blk_mq_add_to_requeue_list directly from the two callers. Signed-off-by:
-
Christoph Hellwig authored
Remove this very small helper and fold it into the only caller. Note that this moves the trace_block_rq_insert out of ctx->lock, matching the other calls to this tracepoint. Signed-off-by:
-
Christoph Hellwig authored
There is no good point in keeping the __blk_mq_insert_request around for two function calls and a singler caller. Signed-off-by:
-
Christoph Hellwig authored
blk_mq_sched_insert_request is the main request insert helper and not directly I/O scheduler related. Move blk_mq_sched_insert_request to blk-mq.c, rename it to blk_mq_insert_request and mark it static. Signed-off-by:
-
Christoph Hellwig authored
blk_mq_dispatch_plug_list is the only caller of blk_mq_sched_insert_requests, and it makes sense to just fold it there as blk_mq_sched_insert_requests isn't specific to I/O schedulers despite the name. Signed-off-by:
-
Christoph Hellwig authored
Move all logic related to the direct insert (including the call to blk_mq_run_hw_queue) into blk_mq_insert_requests to streamline the code flow up a bit, and to allow marking blk_mq_try_issue_list_directly static. Signed-off-by:
-
Christoph Hellwig authored
block/blk-mq.h needs various definitions from <linux/blk-mq.h>, include it there instead of relying on the source files to include both. Signed-off-by:
-
Christoph Hellwig authored
blk-mq-tag.h is always included by blk-mq.h, and causes recursive inclusion hell with further changes. Just merge it into blk-mq.h instead. Signed-off-by:
-
Christoph Hellwig authored
Plugs never insert at head, so don't plug for head insertions. Fixes: 1c2d2fff ("block: wire-up support for passthrough plugging") Signed-off-by:
-
Chengming Zhou authored
blk_throtl_register() will unconditionally enable blk-stat for gendisk when register, even when we have no BLK_DEV_THROTTLING_LOW config. Since the kernel always has only BLK_DEV_THROTTLING config and the BLK_DEV_THROTTLING_LOW config is still in EXPERIMENTAL state, we can just skip blk-stat when !BLK_DEV_THROTTLING_LOW. Signed-off-by:
Chengming Zhou <zhouchengming@bytedance.com> Acked-by:
Tejun Heo <tj@kernel.org> Link: https://lore.kernel.org/r/20230413062805.2081970-2-chengming.zhou@linux.devSigned-off-by:
-
Chengming Zhou authored
We need to set QUEUE_FLAG_STATS for two cases: 1. blk_stat_enable_accounting() 2. blk_stat_add_callback() So we should clear it only when ((q->stats->accounting == 0) && list_empty(&q->stats->callbacks)). blk_stat_disable_accounting() only check if q->stats->accounting is 0 before clear the flag, this patch fix it. Also add list_empty(&q->stats->callbacks)) check when enable, or the flag is already set. The bug can be reproduced on kernel without BLK_DEV_THROTTLING (since it unconditionally enable accounting, see the next patch). # cat /sys/block/sr0/queue/scheduler none mq-deadline [bfq] # cat /sys/kernel/debug/block/sr0/state SAME_COMP|IO_STAT|INIT_DONE|STATS|REGISTERED|NOWAIT|30 # echo none > /sys/block/sr0/queue/scheduler # cat /sys/kernel/debug/block/sr0/state SAME_COMP|IO_STAT|INIT_DONE|REGISTERED|NOWAIT # cat /sys/block/sr0/queue/wbt_lat_usec 75000 We can see that after changing elevator from "bfq" to "none", "STATS" flag is lost even though WBT callback still need it. Fixes: 68497092 ("block: make queue stat accounting a reference") Cc: <stable@vger.kernel.org> # v5.17+ Signed-off-by:
-
Tejun Heo authored
Other rq_qos policies such as wbt and iocost are lazy-initialized when they are configured for the first time for the device but iolatency is initialized unconditionally from blkcg_init_disk() during gendisk init. Lazy init is beneficial because rq_qos policies add runtime overhead when initialized as every IO has to walk all registered rq_qos callbacks. This patch switches iolatency to lazy initialization too so that it only registered its rq_qos policy when it is first configured. Note that there is a known race condition between blkcg config file writes and del_gendisk() and this patch makes iolatency susceptible to it by exposing the init path to race against the deletion path. However, that problem already exists in iocost and is being worked on. Signed-off-by:
-
Tejun Heo authored
The name was too generic given that there are multiple blkcg rq-qos policies. Signed-off-by:
-
Tejun Heo authored
We want to support lazy init of rq-qos policies so that iolatency is enabled lazily on configuration instead of gendisk initialization. The way blkg config helpers are structured now is a bit awkward for that. Let's restructure: * blkcg_conf_open_bdev() is renamed to blkg_conf_open_bdev(). The blkcg_ prefix was used because the bdev opening step is blkg-independent. However, the distinction is too subtle and confuses more than helps. Let's switch to blkg prefix so that it's consistent with the type and other helper names. * struct blkg_conf_ctx now remembers the original input string and is always initialized by the new blkg_conf_init(). * blkg_conf_open_bdev() is updated to take a pointer to blkg_conf_ctx like blkg_conf_prep() and can be called multiple times safely. Instead of modifying the double pointer to input string directly, blkg_conf_open_bdev() now sets blkg_conf_ctx->body. * blkg_conf_finish() is renamed to blkg_conf_exit() for symmetry and now must be called on all blkg_conf_ctx's which were initialized with blkg_conf_init(). Combined, this allows the users to either open the bdev first or do it altogether with blkg_conf_prep() which will help implementing lazy init of rq-qos policies. blkg_conf_init/exit() will also be used implement synchronization against device removal. This is necessary because iolat / iocost are configured through cgroupfs instead of one of the files under /sys/block/DEVICE. As cgroupfs operations aren't synchronized with block layer, the lazy init and other configuration operations may race against device removal. This patch makes blkg_conf_init/exit() used consistently for all cgroup-orginating configurations making them a good place to implement explicit synchronization. Users are updated accordingly. No behavior change is intended by this patch. v2: bfq wasn't updated in v1 causing a build error. Fixed. v3: Update the description to include future use of blkg_conf_init/exit() as synchronization points. Signed-off-by: -
Tejun Heo authored
Now that all RCU flavors have been combined either holding a spin lock, disabling irq or disabling preemption implies RCU read lock, so there's no need to use rcu_read_[un]lock() explicitly while holding queue_lock. This shouldn't cause any behavior changes. v2: Description updated. Leave __acquires/release on queue_lock alone. Signed-off-by:
-
- 12 Apr, 2023 7 commits
-
-
Stefan Haberland authored
The DASD driver does not kick the requeue list when requeuing IO requests to the blocklayer. This might lead to hanging blockdevice when there is no other trigger for this. Fix by automatically kick the requeue list when requeuing DASD requests to the blocklayer. Fixes: e443343e ("s390/dasd: blk-mq conversion") CC: stable@vger.kernel.org # 4.14+ Signed-off-by:
Stefan Haberland <sth@linux.ibm.com> Reviewed-by:
Jan Hoeppner <hoeppner@linux.ibm.com> Reviewed-by:
Halil Pasic <pasic@linux.ibm.com> Link: https://lore.kernel.org/r/20230405142017.2446986-8-sth@linux.ibm.comSigned-off-by:
-
Stefan Haberland authored
Add a check for errors in the start_io function that signal a not working device. Trigger an autoquiesce event in that case. Signed-off-by:
-
Stefan Haberland authored
Add a sysfs attribute aq_timeouts that controls after how many timeouts a autoquiesce event might be triggered. The default value is 32768 which is the maximum number of retries for the DASD device driver DASD_RETRIES_MAX. This means that the timeout trigger will never happen. The default value for DASD retries is 255. Setting the value to below 255 will trigger the timeout autoquiesce event before an IO error is generated. Also add the check for the configured amount of timeouts and trigger an autoquiesce event if exceeded. Signed-off-by:
-
Stefan Haberland authored
Add a sysfs attribute to control if all IO requests will be requeued to the blocklayer in case of an autoquiesce event or not. A value of 1 means that in case of an autoquiesce event all IO requests will be requeued to the blocklayer. A value of 0 means that the device will only be stopped. Signed-off-by:
-
Stefan Haberland authored
Add sysfs attribute that controls the DASD autoquiesce feature. The autoquiesce is disabled when 0 is echoed to the attribute. A value greater than 0 will enable the feature. The aq_mask attribute will accept an unsigned integer and the value will be interpreted as bitmask defining the trigger events that will lead to an automatic quiesce. The following autoquiesce triggers will currently be available: DASD_EER_FATALERROR 1 - any final I/O error DASD_EER_NOPATH 2 - no remaining paths for the device DASD_EER_STATECHANGE 3 - a state change interrupt occurred DASD_EER_PPRCSUSPEND 4 - the device is PPRC suspended DASD_EER_NOSPC 5 - there is no space remaining on an ESE device DASD_EER_TIMEOUT 6 - a certain amount of timeouts occurred DASD_EER_STARTIO 7 - the IO start function encountered an error The currently supported maximum value is 255. Bit 31 is reserved for internal usage. Bit 0 is not used. Example: - deactivate autoquiesce $ echo 0 > /sys/bus/ccw/0.0.1234/aq_mask - enable autoquiesce for FATALERROR, NOPATH and TIMEOUT (0000 0000 0000 0000 0000 0000 0100 0110 => 70) $ echo 70 > /sys/bus/ccw/0.0.1234/aq_mask Signed-off-by:
-
Stefan Haberland authored
Add the internal logic to check for autoquiesce triggers and handle them. Quiesce and resume are functions that tell Linux to stop/resume issuing I/Os to a specific DASD. The DASD driver allows a manual quiesce/resume via ioctl. Autoquiesce will define an amount of triggers that will lead to an automatic quiesce if a certain event occurs. There is no automatic resume. All events will be reported via DASD Extended Error Reporting (EER) if configured. Signed-off-by:
-
Stefan Haberland authored
Remove definitions that have never been used. Signed-off-by:
-
- 06 Apr, 2023 3 commits
-
-
Chengming Zhou authored
blkcg_policy cpd_init_fn() is used to just initialize some default fields of policy data, which is enough to do in cpd_alloc_fn(). This patch delete the only user bfq_cpd_init(), and remove cpd_init_fn from blkcg_policy. Signed-off-by:
-
Chengming Zhou authored
cpd_bind_fn is just used for update default weight when block subsys attached to a hierarchy. No any policy need it anymore. Signed-off-by:
-
Chengming Zhou authored
BFQ_WEIGHT_LEGACY_DFL is the same as CGROUP_WEIGHT_DFL, which means we don't need cpd_bind_fn() callback to update default weight when attached to a hierarchy. This patch remove BFQ_WEIGHT_LEGACY_DFL and cpd_bind_fn(). Signed-off-by:
-
