• Waiman Long's avatar
    locking/rwsem: Make owner store task pointer of last owning reader · 925b9cd1
    Waiman Long authored
    Currently, when a reader acquires a lock, it only sets the
    RWSEM_READER_OWNED bit in the owner field. The other bits are simply
    not used. When debugging hanging cases involving rwsems and readers,
    the owner value does not provide much useful information at all.
    
    This patch modifies the current behavior to always store the task_struct
    pointer of the last rwsem-acquiring reader in a reader-owned rwsem. This
    may be useful in debugging rwsem hanging cases especially if only one
    reader is involved. However, the task in the owner field may not the
    real owner or one of the real owners at all when the owner value is
    examined, for example, in a crash dump. So it is just an additional
    hint about the past history.
    
    If CONFIG_DEBUG_RWSEMS=y is enabled, the owner field will be checked at
    unlock time too to make sure the task pointer value is valid. That does
    have a slight performance cost and so is only enabled as part of that
    debug option.
    
    From the performance point of view, it is expected that the changes
    shouldn't have any noticeable performance impact. A rwsem microbenchmark
    (with 48 worker threads and 1:1 reader/writer ratio) was ran on a
    2-socket 24-core 48-thread Haswell system.  The locking rates on a
    4.19-rc1 based kernel were as follows:
    
      1) Unpatched kernel:				543.3 kops/s
      2) Patched kernel:				549.2 kops/s
      3) Patched kernel (CONFIG_DEBUG_RWSEMS on):	546.6 kops/s
    
    There was actually a slight increase in performance (1.1%) in this
    particular case. Maybe it was caused by the elimination of a branch or
    just a testing noise. Turning on the CONFIG_DEBUG_RWSEMS option also
    had less than the expected impact on performance.
    
    The least significant 2 bits of the owner value are now used to designate
    the rwsem is readers owned and the owners are anonymous.
    Signed-off-by: default avatarWaiman Long <longman@redhat.com>
    Acked-by: default avatarPeter Zijlstra <a.p.zijlstra@chello.nl>
    Cc: Davidlohr Bueso <dave@stgolabs.net>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: Will Deacon <will.deacon@arm.com>
    Link: http://lkml.kernel.org/r/1536265114-10842-1-git-send-email-longman@redhat.comSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
    925b9cd1
rwsem-xadd.c 19.9 KB