• Waiman Long's avatar
    locking/pvqspinlock: Extend node size when pvqspinlock is configured · 0fa809ca
    Waiman Long authored
    The qspinlock code supports up to 4 levels of slowpath nesting using
    four per-CPU mcs_spinlock structures. For 64-bit architectures, they
    fit nicely in one 64-byte cacheline.
    
    For para-virtualized (PV) qspinlocks it needs to store more information
    in the per-CPU node structure than there is space for. It uses a trick
    to use a second cacheline to hold the extra information that it needs.
    So PV qspinlock needs to access two extra cachelines for its information
    whereas the native qspinlock code only needs one extra cacheline.
    
    Freshly added counter profiling of the qspinlock code, however, revealed
    that it was very rare to use more than two levels of slowpath nesting.
    So it doesn't make sense to penalize PV qspinlock code in order to have
    four mcs_spinlock structures in the same cacheline to optimize for a case
    in the native qspinlock code that rarely happens.
    
    Extend the per-CPU node structure to have two more long words when PV
    qspinlock locks are configured to hold the extra data that it needs.
    
    As a result, the PV qspinlock code will enjoy the same benefit of using
    just one extra cacheline like the native counterpart, for most cases.
    
    [ mingo: Minor changelog edits. ]
    Signed-off-by: default avatarWaiman Long <longman@redhat.com>
    Cc: Andrew Morton <akpm@linux-foundation.org>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: Will Deacon <will.deacon@arm.com>
    Link: http://lkml.kernel.org/r/1539697507-28084-2-git-send-email-longman@redhat.comSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
    0fa809ca
qspinlock.c 16.3 KB