• Paul E. McKenney's avatar
    rcu: Accelerate grace period if last non-dynticked CPU · 8bd93a2c
    Paul E. McKenney authored
    Currently, rcu_needs_cpu() simply checks whether the current CPU
    has an outstanding RCU callback, which means that the last CPU
    to go into dyntick-idle mode might wait a few ticks for the
    relevant grace periods to complete.  However, if all the other
    CPUs are in dyntick-idle mode, and if this CPU is in a quiescent
    state (which it is for RCU-bh and RCU-sched any time that we are
    considering going into dyntick-idle mode), then the grace period
    is instantly complete.
    
    This patch therefore repeatedly invokes the RCU grace-period
    machinery in order to force any needed grace periods to complete
    quickly.  It does so a limited number of times in order to
    prevent starvation by an RCU callback function that might pass
    itself to call_rcu().
    
    However, if any CPU other than the current one is not in
    dyntick-idle mode, fall back to simply checking (with fix to bug
    noted by Lai Jiangshan).  Also, take advantage of last
    grace-period forcing, the opportunity to do so noted by Steve
    Rostedt.  And apply simplified #ifdef condition suggested by
    Frederic Weisbecker.
    Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
    Cc: laijs@cn.fujitsu.com
    Cc: dipankar@in.ibm.com
    Cc: mathieu.desnoyers@polymtl.ca
    Cc: josh@joshtriplett.org
    Cc: dvhltc@us.ibm.com
    Cc: niv@us.ibm.com
    Cc: peterz@infradead.org
    Cc: rostedt@goodmis.org
    Cc: Valdis.Kletnieks@vt.edu
    Cc: dhowells@redhat.com
    LKML-Reference: <1266887105-1528-15-git-send-email-paulmck@linux.vnet.ibm.com>
    Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
    8bd93a2c
rcutree.c 55.4 KB