• Thomas Gleixner's avatar
    timer: Minimize nohz off overhead · 683be13a
    Thomas Gleixner authored
    If nohz is disabled on the kernel command line the [hr]timer code
    still calls wake_up_nohz_cpu() and tick_nohz_full_cpu(), a pretty
    pointless exercise. Cache nohz_active in [hr]timer per cpu bases and
    avoid the overhead.
    
    Before:
      48.10%  hog       [.] main
      15.25%  [kernel]  [k] _raw_spin_lock_irqsave
       9.76%  [kernel]  [k] _raw_spin_unlock_irqrestore
       6.50%  [kernel]  [k] mod_timer
       6.44%  [kernel]  [k] lock_timer_base.isra.38
       3.87%  [kernel]  [k] detach_if_pending
       3.80%  [kernel]  [k] del_timer
       2.67%  [kernel]  [k] internal_add_timer
       1.33%  [kernel]  [k] __internal_add_timer
       0.73%  [kernel]  [k] timerfn
       0.54%  [kernel]  [k] wake_up_nohz_cpu
    
    After:
      48.73%  hog       [.] main
      15.36%  [kernel]  [k] _raw_spin_lock_irqsave
       9.77%  [kernel]  [k] _raw_spin_unlock_irqrestore
       6.61%  [kernel]  [k] lock_timer_base.isra.38
       6.42%  [kernel]  [k] mod_timer
       3.90%  [kernel]  [k] detach_if_pending
       3.76%  [kernel]  [k] del_timer
       2.41%  [kernel]  [k] internal_add_timer
       1.39%  [kernel]  [k] __internal_add_timer
       0.76%  [kernel]  [k] timerfn
    
    We probably should have a cached value for nohz full in the per cpu
    bases as well to avoid the cpumask check. The base cache line is hot
    already, the cpumask not necessarily.
    Signed-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
    Cc: Frederic Weisbecker <fweisbec@gmail.com>
    Cc: Eric Dumazet <edumazet@google.com>
    Cc: Viresh Kumar <viresh.kumar@linaro.org>
    Cc: John Stultz <john.stultz@linaro.org>
    Cc: Joonwoo Park <joonwoop@codeaurora.org>
    Cc: Wenbo Wang <wenbo.wang@memblaze.com>
    Link: http://lkml.kernel.org/r/20150526224512.207378134@linutronix.deSigned-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
    683be13a
hrtimer.c 45.9 KB