• Paul E. McKenney's avatar
    rcu: Make rcu_read_unlock_special() checks match raise_softirq_irqoff() · 87446b48
    Paul E. McKenney authored
    Threaded interrupts provide additional interesting interactions between
    RCU and raise_softirq() that can result in self-deadlocks in v5.0-2 of
    the Linux kernel.  These self-deadlocks can be provoked in susceptible
    kernels within a few minutes using the following rcutorture command on
    an 8-CPU system:
    
    tools/testing/selftests/rcutorture/bin/kvm.sh --duration 5 --configs "TREE03" --bootargs "threadirqs"
    
    Although post-v5.2 RCU commits have at least greatly reduced the
    probability of these self-deadlocks, this was entirely by accident.
    Although this sort of accident should be rowdily celebrated on those
    rare occasions when it does occur, such celebrations should be quickly
    followed by a principled patch, which is what this patch purports to be.
    
    The key point behind this patch is that when in_interrupt() returns
    true, __raise_softirq_irqoff() will never attempt a wakeup.  Therefore,
    if in_interrupt(), calls to raise_softirq*() are both safe and
    extremely cheap.
    
    This commit therefore replaces the in_irq() calls in the "if" statement
    in rcu_read_unlock_special() with in_interrupt() and simplifies the
    "if" condition to the following:
    
    	if (irqs_were_disabled && use_softirq &&
    	    (in_interrupt() ||
    	     (exp && !t->rcu_read_unlock_special.b.deferred_qs))) {
    		raise_softirq_irqoff(RCU_SOFTIRQ);
    	} else {
    		/* Appeal to the scheduler. */
    	}
    
    The rationale behind the "if" condition is as follows:
    
    1.	irqs_were_disabled:  If interrupts are enabled, we should
    	instead appeal to the scheduler so as to let the upcoming
    	irq_enable()/local_bh_enable() do the rescheduling for us.
    2.	use_softirq: If this kernel isn't using softirq, then
    	raise_softirq_irqoff() will be unhelpful.
    3.	a.	in_interrupt(): If this returns true, the subsequent
    		call to raise_softirq_irqoff() is guaranteed not to
    		do a wakeup, so that call will be both very cheap and
    		quite safe.
    	b.	Otherwise, if !in_interrupt() the raise_softirq_irqoff()
    		might do a wakeup, which is expensive and, in some
    		contexts, unsafe.
    		i.	The "exp" (an expedited RCU grace period is being
    			blocked) says that the wakeup is worthwhile, and:
    		ii.	The !.deferred_qs says that scheduler locks
    			cannot be held, so the wakeup will be safe.
    
    Backporting this requires considerable care, so no auto-backport, please!
    
    Fixes: 05f41571 ("rcu: Speed up expedited GPs when interrupting RCU reader")
    Reported-by: default avatarSebastian Andrzej Siewior <bigeasy@linutronix.de>
    Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
    87446b48
tree_plugin.h 72.5 KB