• Peter Zijlstra's avatar
    sched/idle: Optimize try-to-wake-up IPI · e3baac47
    Peter Zijlstra authored
    [ This series reduces the number of IPIs on Andy's workload by something like
      99%. It's down from many hundreds per second to very few.
    
      The basic idea behind this series is to make TIF_POLLING_NRFLAG be a
      reliable indication that the idle task is polling.  Once that's done,
      the rest is reasonably straightforward. ]
    
    When enqueueing tasks on remote LLC domains, we send an IPI to do the
    work 'locally' and avoid bouncing all the cachelines over.
    
    However, when the remote CPU is idle (and polling, say x86 mwait), we
    don't need to send an IPI, we can simply kick the TIF word to wake it
    up and have the 'idle' loop do the work.
    
    So when _TIF_POLLING_NRFLAG is set, but _TIF_NEED_RESCHED is not (yet)
    set, set _TIF_NEED_RESCHED and avoid sending the IPI.
    Much-requested-by: default avatarAndy Lutomirski <luto@amacapital.net>
    Signed-off-by: default avatarPeter Zijlstra <peterz@infradead.org>
    [Edited by Andy Lutomirski, but this is mostly Peter Zijlstra's code.]
    Signed-off-by: default avatarAndy Lutomirski <luto@amacapital.net>
    Cc: nicolas.pitre@linaro.org
    Cc: daniel.lezcano@linaro.org
    Cc: Mike Galbraith <umgwanakikbuti@gmail.com>
    Cc: umgwanakikbuti@gmail.com
    Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: linux-kernel@vger.kernel.org
    Link: http://lkml.kernel.org/r/ce06f8b02e7e337be63e97597fc4b248d3aa6f9b.1401902905.git.luto@amacapital.netSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
    e3baac47
idle.c 6.48 KB