• Peter Zijlstra's avatar
    locking: Introduce smp_mb__after_spinlock() · d89e588c
    Peter Zijlstra authored
    Since its inception, our understanding of ACQUIRE, esp. as applied to
    spinlocks, has changed somewhat. Also, I wonder if, with a simple
    change, we cannot make it provide more.
    
    The problem with the comment is that the STORE done by spin_lock isn't
    itself ordered by the ACQUIRE, and therefore a later LOAD can pass over
    it and cross with any prior STORE, rendering the default WMB
    insufficient (pointed out by Alan).
    
    Now, this is only really a problem on PowerPC and ARM64, both of
    which already defined smp_mb__before_spinlock() as a smp_mb().
    
    At the same time, we can get a much stronger construct if we place
    that same barrier _inside_ the spin_lock(). In that case we upgrade
    the RCpc spinlock to an RCsc.  That would make all schedule() calls
    fully transitive against one another.
    Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
    Acked-by: default avatarWill Deacon <will.deacon@arm.com>
    Cc: Alan Stern <stern@rowland.harvard.edu>
    Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Michael Ellerman <mpe@ellerman.id.au>
    Cc: Nicholas Piggin <npiggin@gmail.com>
    Cc: Oleg Nesterov <oleg@redhat.com>
    Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
    d89e588c
core.c 165 KB