• Jens Axboe's avatar
    io_uring/sqpoll: close race on waiting for sqring entries · 28aabffa
    Jens Axboe authored
    When an application uses SQPOLL, it must wait for the SQPOLL thread to
    consume SQE entries, if it fails to get an sqe when calling
    io_uring_get_sqe(). It can do so by calling io_uring_enter(2) with the
    flag value of IORING_ENTER_SQ_WAIT. In liburing, this is generally done
    with io_uring_sqring_wait(). There's a natural expectation that once
    this call returns, a new SQE entry can be retrieved, filled out, and
    submitted. However, the kernel uses the cached sq head to determine if
    the SQRING is full or not. If the SQPOLL thread is currently in the
    process of submitting SQE entries, it may have updated the cached sq
    head, but not yet committed it to the SQ ring. Hence the kernel may find
    that there are SQE entries ready to be consumed, and return successfully
    to the application. If the SQPOLL thread hasn't yet committed the SQ
    ring entries by the time the application returns to userspace and
    attempts to get a new SQE, it will fail getting a new SQE.
    
    Fix this by having io_sqring_full() always use the user visible SQ ring
    head entry, rather than the internally cached one.
    
    Cc: stable@vger.kernel.org # 5.10+
    Link: https://github.com/axboe/liburing/discussions/1267Reported-by: default avatarBenedek Thaler <thaler@thaler.hu>
    Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
    28aabffa
io_uring.h 13.3 KB