• Johannes Weiner's avatar
    wait: prevent exclusive waiter starvation · 777c6c5f
    Johannes Weiner authored
    With exclusive waiters, every process woken up through the wait queue must
    ensure that the next waiter down the line is woken when it has finished.
    
    Interruptible waiters don't do that when aborting due to a signal.  And if
    an aborting waiter is concurrently woken up through the waitqueue, noone
    will ever wake up the next waiter.
    
    This has been observed with __wait_on_bit_lock() used by
    lock_page_killable(): the first contender on the queue was aborting when
    the actual lock holder woke it up concurrently.  The aborted contender
    didn't acquire the lock and therefor never did an unlock followed by
    waking up the next waiter.
    
    Add abort_exclusive_wait() which removes the process' wait descriptor from
    the waitqueue, iff still queued, or wakes up the next waiter otherwise.
    It does so under the waitqueue lock.  Racing with a wake up means the
    aborting process is either already woken (removed from the queue) and will
    wake up the next waiter, or it will remove itself from the queue and the
    concurrent wake up will apply to the next waiter after it.
    
    Use abort_exclusive_wait() in __wait_event_interruptible_exclusive() and
    __wait_on_bit_lock() when they were interrupted by other means than a wake
    up through the queue.
    
    [akpm@linux-foundation.org: coding-style fixes]
    Reported-by: default avatarChris Mason <chris.mason@oracle.com>
    Signed-off-by: default avatarJohannes Weiner <hannes@cmpxchg.org>
    Mentored-by: default avatarOleg Nesterov <oleg@redhat.com>
    Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
    Cc: Matthew Wilcox <matthew@wil.cx>
    Cc: Chuck Lever <cel@citi.umich.edu>
    Cc: Nick Piggin <nickpiggin@yahoo.com.au>
    Cc: Ingo Molnar <mingo@elte.hu>
    Cc: <stable@kernel.org>		["after some testing"]
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    777c6c5f
wait.c 8.15 KB