• Linus Torvalds's avatar
    Merge tag 'locking-core-2021-08-30' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip · e5e726f7
    Linus Torvalds authored
    Pull locking and atomics updates from Thomas Gleixner:
     "The regular pile:
    
       - A few improvements to the mutex code
    
       - Documentation updates for atomics to clarify the difference between
         cmpxchg() and try_cmpxchg() and to explain the forward progress
         expectations.
    
       - Simplification of the atomics fallback generator
    
       - The addition of arch_atomic_long*() variants and generic arch_*()
         bitops based on them.
    
       - Add the missing might_sleep() invocations to the down*() operations
         of semaphores.
    
      The PREEMPT_RT locking core:
    
       - Scheduler updates to support the state preserving mechanism for
         'sleeping' spin- and rwlocks on RT.
    
         This mechanism is carefully preserving the state of the task when
         blocking on a 'sleeping' spin- or rwlock and takes regular wake-ups
         targeted at the same task into account. The preserved or updated
         (via a regular wakeup) state is restored when the lock has been
         acquired.
    
       - Restructuring of the rtmutex code so it can be utilized and
         extended for the RT specific lock variants.
    
       - Restructuring of the ww_mutex code to allow sharing of the ww_mutex
         specific functionality for rtmutex based ww_mutexes.
    
       - Header file disentangling to allow substitution of the regular lock
         implementations with the PREEMPT_RT variants without creating an
         unmaintainable #ifdef mess.
    
       - Shared base code for the PREEMPT_RT specific rw_semaphore and
         rwlock implementations.
    
         Contrary to the regular rw_semaphores and rwlocks the PREEMPT_RT
         implementation is writer unfair because it is infeasible to do
         priority inheritance on multiple readers. Experience over the years
         has shown that real-time workloads are not the typical workloads
         which are sensitive to writer starvation.
    
         The alternative solution would be to allow only a single reader
         which has been tried and discarded as it is a major bottleneck
         especially for mmap_sem. Aside of that many of the writer
         starvation critical usage sites have been converted to a writer
         side mutex/spinlock and RCU read side protections in the past
         decade so that the issue is less prominent than it used to be.
    
       - The actual rtmutex based lock substitutions for PREEMPT_RT enabled
         kernels which affect mutex, ww_mutex, rw_semaphore, spinlock_t and
         rwlock_t. The spin/rw_lock*() functions disable migration across
         the critical section to preserve the existing semantics vs per-CPU
         variables.
    
       - Rework of the futex REQUEUE_PI mechanism to handle the case of
         early wake-ups which interleave with a re-queue operation to
         prevent the situation that a task would be blocked on both the
         rtmutex associated to the outer futex and the rtmutex based hash
         bucket spinlock.
    
         While this situation cannot happen on !RT enabled kernels the
         changes make the underlying concurrency problems easier to
         understand in general. As a result the difference between !RT and
         RT kernels is reduced to the handling of waiting for the critical
         section. !RT kernels simply spin-wait as before and RT kernels
         utilize rcu_wait().
    
       - The substitution of local_lock for PREEMPT_RT with a spinlock which
         protects the critical section while staying preemptible. The CPU
         locality is established by disabling migration.
    
      The underlying concepts of this code have been in use in PREEMPT_RT for
      way more than a decade. The code has been refactored several times over
      the years and this final incarnation has been optimized once again to be
      as non-intrusive as possible, i.e. the RT specific parts are mostly
      isolated.
    
      It has been extensively tested in the 5.14-rt patch series and it has
      been verified that !RT kernels are not affected by these changes"
    
    * tag 'locking-core-2021-08-30' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (92 commits)
      locking/rtmutex: Return success on deadlock for ww_mutex waiters
      locking/rtmutex: Prevent spurious EDEADLK return caused by ww_mutexes
      locking/rtmutex: Dequeue waiter on ww_mutex deadlock
      locking/rtmutex: Dont dereference waiter lockless
      locking/semaphore: Add might_sleep() to down_*() family
      locking/ww_mutex: Initialize waiter.ww_ctx properly
      static_call: Update API documentation
      locking/local_lock: Add PREEMPT_RT support
      locking/spinlock/rt: Prepare for RT local_lock
      locking/rtmutex: Add adaptive spinwait mechanism
      locking/rtmutex: Implement equal priority lock stealing
      preempt: Adjust PREEMPT_LOCK_OFFSET for RT
      locking/rtmutex: Prevent lockdep false positive with PI futexes
      futex: Prevent requeue_pi() lock nesting issue on RT
      futex: Simplify handle_early_requeue_pi_wakeup()
      futex: Reorder sanity checks in futex_requeue()
      futex: Clarify comment in futex_requeue()
      futex: Restructure futex_requeue()
      futex: Correct the number of requeued waiters for PI
      futex: Remove bogus condition for requeue PI
      ...
    e5e726f7
tree_plugin.h 48 KB