• Peter Zijlstra's avatar
    locking/mutex: Fix mutex handoff · e274795e
    Peter Zijlstra authored
    While reviewing the ww_mutex patches, I noticed that it was still
    possible to (incorrectly) succeed for (incorrect) code like:
    
    	mutex_lock(&a);
    	mutex_lock(&a);
    
    This was possible if the second mutex_lock() would block (as expected)
    but then receive a spurious wakeup. At that point it would find itself
    at the front of the queue, request a handoff and instantly claim
    ownership and continue, since owner would point to itself.
    
    Avoid this scenario and simplify the code by introducing a third low
    bit to signal handoff pickup. So once we request handoff, unlock
    clears the handoff bit and sets the pickup bit along with the new
    owner.
    
    This also removes the need for the .handoff argument to
    __mutex_trylock(), since that becomes superfluous with PICKUP.
    
    In order to guarantee enough low bits, ensure task_struct alignment is
    at least L1_CACHE_BYTES (which seems a good ideal regardless).
    Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
    Cc: Andrew Morton <akpm@linux-foundation.org>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Fixes: 9d659ae1 ("locking/mutex: Add lock handoff to avoid starvation")
    Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
    e274795e
fork.c 55.9 KB