• Paul E. McKenney's avatar
    rcu: Fix synchronization for rcu_process_gp_end() uses of ->completed counter · d09b62df
    Paul E. McKenney authored
    Impose a clear locking design on the rcu_process_gp_end()
    function's use of the ->completed counter.  This is done by
    creating a ->completed field in the rcu_node structure, which
    can safely be accessed under the protection of that structure's
    lock.  Performance and scalability are maintained by using a
    form of double-checked locking, so that rcu_process_gp_end()
    only acquires the leaf rcu_node structure's ->lock if a grace
    period has recently ended.
    
    This fix reduces rcutorture failure rate by at least two orders
    of magnitude under heavy stress with force_quiescent_state()
    being invoked artificially often.  Without this fix,
    unsynchronized access to the ->completed field can cause
    rcu_process_gp_end() to advance callbacks whose grace period has
    not yet expired.  (Bad idea!)
    Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
    Cc: laijs@cn.fujitsu.com
    Cc: dipankar@in.ibm.com
    Cc: mathieu.desnoyers@polymtl.ca
    Cc: josh@joshtriplett.org
    Cc: dvhltc@us.ibm.com
    Cc: niv@us.ibm.com
    Cc: peterz@infradead.org
    Cc: rostedt@goodmis.org
    Cc: Valdis.Kletnieks@vt.edu
    Cc: dhowells@redhat.com
    Cc: <stable@kernel.org> # .32.x
    LKML-Reference: <12571987494069-git-send-email->
    Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
    d09b62df
rcutree.c 50.6 KB