Commit e9979b36 authored by Pavel Begunkov's avatar Pavel Begunkov Committed by Jens Axboe

io_uring: skip futile iopoll iterations

The only way to get out of io_iopoll_getevents() and continue iterating
is to have empty iopoll_list, otherwise the main loop would just exit.
So, instead of the unlock on 8th time heuristic, do that based on
iopoll_list.

Also, as no one can add new requests to iopoll_list while
io_iopoll_check() hold uring_lock, it's useless to spin with the list
empty, return in that case.
Signed-off-by: default avatarPavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/5b8ebe84f5fff7ffa1f708952dfef7fc78b668e2.1618278933.git.asml.silence@gmail.comSigned-off-by: default avatarJens Axboe <axboe@kernel.dk>
parent cce4b8b0
...@@ -2385,7 +2385,7 @@ static void io_iopoll_try_reap_events(struct io_ring_ctx *ctx) ...@@ -2385,7 +2385,7 @@ static void io_iopoll_try_reap_events(struct io_ring_ctx *ctx)
static int io_iopoll_check(struct io_ring_ctx *ctx, long min) static int io_iopoll_check(struct io_ring_ctx *ctx, long min)
{ {
unsigned int nr_events = 0; unsigned int nr_events = 0;
int iters = 0, ret = 0; int ret = 0;
/* /*
* We disallow the app entering submit/complete with polling, but we * We disallow the app entering submit/complete with polling, but we
...@@ -2414,10 +2414,13 @@ static int io_iopoll_check(struct io_ring_ctx *ctx, long min) ...@@ -2414,10 +2414,13 @@ static int io_iopoll_check(struct io_ring_ctx *ctx, long min)
* forever, while the workqueue is stuck trying to acquire the * forever, while the workqueue is stuck trying to acquire the
* very same mutex. * very same mutex.
*/ */
if (!(++iters & 7)) { if (list_empty(&ctx->iopoll_list)) {
mutex_unlock(&ctx->uring_lock); mutex_unlock(&ctx->uring_lock);
io_run_task_work(); io_run_task_work();
mutex_lock(&ctx->uring_lock); mutex_lock(&ctx->uring_lock);
if (list_empty(&ctx->iopoll_list))
break;
} }
ret = io_iopoll_getevents(ctx, &nr_events, min); ret = io_iopoll_getevents(ctx, &nr_events, min);
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment