• Thomas Gleixner's avatar
    rcu/nocb: Make rcu_core() callbacks acceleration preempt-safe · 24ee940d
    Thomas Gleixner authored
    While reporting a quiescent state for a given CPU, rcu_core() takes
    advantage of the freshly loaded grace period sequence number and the
    locked rnp to accelerate the callbacks whose sequence number have been
    assigned a stale value.
    
    This action is only necessary when the rdp isn't offloaded, otherwise
    the NOCB kthreads already take care of the callbacks progression.
    
    However the check for the offloaded state is volatile because it is
    performed outside the IRQs disabled section. It's possible for the
    offloading process to preempt rcu_core() at that point on PREEMPT_RT.
    
    This is dangerous because rcu_core() may end up accelerating callbacks
    concurrently with NOCB kthreads without appropriate locking.
    
    Fix this with moving the offloaded check inside the rnp locking section.
    Reported-and-tested-by: default avatarValentin Schneider <valentin.schneider@arm.com>
    Reviewed-by: default avatarValentin Schneider <valentin.schneider@arm.com>
    Tested-by: default avatarSebastian Andrzej Siewior <bigeasy@linutronix.de>
    Signed-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
    Cc: Josh Triplett <josh@joshtriplett.org>
    Cc: Joel Fernandes <joel@joelfernandes.org>
    Cc: Boqun Feng <boqun.feng@gmail.com>
    Cc: Neeraj Upadhyay <neeraju@codeaurora.org>
    Cc: Uladzislau Rezki <urezki@gmail.com>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Signed-off-by: default avatarFrederic Weisbecker <frederic@kernel.org>
    Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
    24ee940d
tree.c 153 KB