• Peter Zijlstra's avatar
    sched/fair: Rework sched_fair time accounting · 9dbdb155
    Peter Zijlstra authored
    Christian suffers from a bad BIOS that wrecks his i5's TSC sync. This
    results in him occasionally seeing time going backwards - which
    crashes the scheduler ...
    
    Most of our time accounting can actually handle that except the most
    common one; the tick time update of sched_fair.
    
    There is a further problem with that code; previously we assumed that
    because we get a tick every TICK_NSEC our time delta could never
    exceed 32bits and math was simpler.
    
    However, ever since Frederic managed to get NO_HZ_FULL merged; this is
    no longer the case since now a task can run for a long time indeed
    without getting a tick. It only takes about ~4.2 seconds to overflow
    our u32 in nanoseconds.
    
    This means we not only need to better deal with time going backwards;
    but also means we need to be able to deal with large deltas.
    
    This patch reworks the entire code and uses mul_u64_u32_shr() as
    proposed by Andy a long while ago.
    
    We express our virtual time scale factor in a u32 multiplier and shift
    right and the 32bit mul_u64_u32_shr() implementation reduces to a
    single 32x32->64 multiply if the time delta is still short (common
    case).
    
    For 64bit a 64x64->128 multiply can be used if ARCH_SUPPORTS_INT128.
    Reported-and-Tested-by: default avatarChristian Engelmayer <cengelma@gmx.at>
    Signed-off-by: default avatarPeter Zijlstra <peterz@infradead.org>
    Cc: fweisbec@gmail.com
    Cc: Paul Turner <pjt@google.com>
    Cc: Stanislaw Gruszka <sgruszka@redhat.com>
    Cc: Andy Lutomirski <luto@amacapital.net>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Andrew Morton <akpm@linux-foundation.org>
    Link: http://lkml.kernel.org/r/20131118172706.GI3866@twins.programming.kicks-ass.netSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
    9dbdb155
fair.c 188 KB