• Michel Lespinasse's avatar
    rwsem: lighter active count checks when waking up readers · 70bdc6e0
    Michel Lespinasse authored
    In __rwsem_do_wake(), we can skip the active count check unless we come
    there from up_xxxx().  Also when checking the active count, it is not
    actually necessary to increment it; this allows us to get rid of the read
    side undo code and simplify the calculation of the final rwsem count
    adjustment once we've counted the reader threads to wake.
    
    The basic observation is the following.  When there are waiter threads on
    a rwsem and the spinlock is held, other threads can only increment the
    active count by trying to grab the rwsem in down_xxxx().  However
    down_xxxx() will notice there are waiter threads and take the down_failed
    path, blocking to acquire the spinlock on the way there.  Therefore, a
    thread observing an active count of zero with waiters queued and the
    spinlock held, is protected against other threads acquiring the rwsem
    until it wakes the last waiter or releases the spinlock.
    Signed-off-by: default avatarMichel Lespinasse <walken@google.com>
    Acked-by: default avatarDavid Howells <dhowells@redhat.com>
    Cc: Mike Waychison <mikew@google.com>
    Cc: Suleiman Souhlal <suleiman@google.com>
    Cc: Ying Han <yinghan@google.com>
    Cc: Ingo Molnar <mingo@elte.hu>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: "H. Peter Anvin" <hpa@zytor.com>
    Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    70bdc6e0
rwsem.c 7.41 KB