• Paul E. McKenney's avatar
    srcu: Expedited grace periods with reduced memory contention · 1e9a038b
    Paul E. McKenney authored
    Commit f60d231a ("srcu: Crude control of expedited grace periods")
    introduced a per-srcu_struct atomic counter to track outstanding
    requests for grace periods.  This works, but represents a memory-contention
    bottleneck.  This commit therefore uses the srcu_node combining tree
    to remove this bottleneck.
    
    This commit adds new ->srcu_gp_seq_needed_exp fields to the
    srcu_data, srcu_node, and srcu_struct structures, which track the
    farthest-in-the-future grace period that must be expedited, which in
    turn requires that all nearer-term grace periods also be expedited.
    Requests for expediting start with the srcu_data structure, run up
    through the srcu_node tree, and end at the srcu_struct structure.
    Note that it may be necessary to expedite a grace period that just
    now started, and this is handled by a new srcu_funnel_exp_start()
    function, which is invoked when the grace period itself is already
    in its way, but when that grace period was not marked as expedited.
    
    A new srcu_get_delay() function returns zero if there is at least one
    expedited SRCU grace period in flight, or SRCU_INTERVAL otherwise.
    This function is used to calculate delays:  Normal grace periods
    are allowed to extend in order to cover more requests with a given
    grace-period computation, which decreases per-request overhead.
    Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
    Tested-by: default avatarMike Galbraith <efault@gmx.de>
    1e9a038b
srcutree.c 35 KB