1. 28 Oct, 2021 1 commit
    • Seth Forshee's avatar
      net: sch: eliminate unnecessary RCU waits in mini_qdisc_pair_swap() · 26746382
      Seth Forshee authored
      Currently rcu_barrier() is used to ensure that no readers of the
      inactive mini_Qdisc buffer remain before it is reused. This waits for
      any pending RCU callbacks to complete, when all that is actually
      required is to wait for one RCU grace period to elapse after the buffer
      was made inactive. This means that using rcu_barrier() may result in
      unnecessary waits.
      
      To improve this, store the current RCU state when a buffer is made
      inactive and use poll_state_synchronize_rcu() to check whether a full
      grace period has elapsed before reusing it. If a full grace period has
      not elapsed, wait for a grace period to elapse, and in the non-RT case
      use synchronize_rcu_expedited() to hasten it.
      
      Since this approach eliminates the RCU callback it is no longer
      necessary to synchronize_rcu() in the tp_head==NULL case. However, the
      RCU state should still be saved for the previously active buffer.
      
      Before this change I would typically see mini_qdisc_pair_swap() take
      tens of milliseconds to complete. After this change it typcially
      finishes in less than 1 ms, and often it takes just a few microseconds.
      
      Thanks to Paul for walking me through the options for improving this.
      
      Cc: "Paul E. McKenney" <paulmck@kernel.org>
      Signed-off-by: default avatarSeth Forshee <sforshee@digitalocean.com>
      Link: https://lore.kernel.org/r/20211026130700.121189-1-seth@forshee.meSigned-off-by: default avatarJakub Kicinski <kuba@kernel.org>
      26746382
  2. 27 Oct, 2021 32 commits
  3. 26 Oct, 2021 7 commits