• Peter Zijlstra's avatar
    locking/static_keys: Add a new static_key interface · 11276d53
    Peter Zijlstra authored
    There are various problems and short-comings with the current
    static_key interface:
    
     - static_key_{true,false}() read like a branch depending on the key
       value, instead of the actual likely/unlikely branch depending on
       init value.
    
     - static_key_{true,false}() are, as stated above, tied to the
       static_key init values STATIC_KEY_INIT_{TRUE,FALSE}.
    
     - we're limited to the 2 (out of 4) possible options that compile to
       a default NOP because that's what our arch_static_branch() assembly
       emits.
    
    So provide a new static_key interface:
    
      DEFINE_STATIC_KEY_TRUE(name);
      DEFINE_STATIC_KEY_FALSE(name);
    
    Which define a key of different types with an initial true/false
    value.
    
    Then allow:
    
       static_branch_likely()
       static_branch_unlikely()
    
    to take a key of either type and emit the right instruction for the
    case.
    
    This means adding a second arch_static_branch_jump() assembly helper
    which emits a JMP per default.
    
    In order to determine the right instruction for the right state,
    encode the branch type in the LSB of jump_entry::key.
    
    This is the final step in removing the naming confusion that has led to
    a stream of avoidable bugs such as:
    
      a833581e ("x86, perf: Fix static_key bug in load_mm_cr4()")
    
    ... but it also allows new static key combinations that will give us
    performance enhancements in the subsequent patches.
    
    Tested-by: Rabin Vincent <rabin@rab.in> # arm
    Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
    Acked-by: Michael Ellerman <mpe@ellerman.id.au> # ppc
    Acked-by: Heiko Carstens <heiko.carstens@de.ibm.com> # s390
    Cc: Andrew Morton <akpm@linux-foundation.org>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: linux-kernel@vger.kernel.org
    Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
    11276d53
jump_label.h 970 Bytes