• Johannes Weiner's avatar
    sched: psi: fix bogus pressure spikes from aggregation race · 3840cbe2
    Johannes Weiner authored
    Brandon reports sporadic, non-sensical spikes in cumulative pressure
    time (total=) when reading cpu.pressure at a high rate. This is due to
    a race condition between reader aggregation and tasks changing states.
    
    While it affects all states and all resources captured by PSI, in
    practice it most likely triggers with CPU pressure, since scheduling
    events are so frequent compared to other resource events.
    
    The race context is the live snooping of ongoing stalls during a
    pressure read. The read aggregates per-cpu records for stalls that
    have concluded, but will also incorporate ad-hoc the duration of any
    active state that hasn't been recorded yet. This is important to get
    timely measurements of ongoing stalls. Those ad-hoc samples are
    calculated on-the-fly up to the current time on that CPU; since the
    stall hasn't concluded, it's expected that this is the minimum amount
    of stall time that will enter the per-cpu records once it does.
    
    The problem is that the path that concludes the state uses a CPU clock
    read that is not synchronized against aggregators; the clock is read
    outside of the seqlock protection. This allows aggregators to race and
    snoop a stall with a longer duration than will actually be recorded.
    
    With the recorded stall time being less than the last snapshot
    remembered by the aggregator, a subsequent sample will underflow and
    observe a bogus delta value, resulting in an erratic jump in pressure.
    
    Fix this by moving the clock read of the state change into the seqlock
    protection. This ensures no aggregation can snoop live stalls past the
    time that's recorded when the state concludes.
    Reported-by: default avatarBrandon Duffany <brandon@buildbuddy.io>
    Link: https://bugzilla.kernel.org/show_bug.cgi?id=219194
    Link: https://lore.kernel.org/lkml/20240827121851.GB438928@cmpxchg.org/
    Fixes: df774306 ("psi: Reduce calls to sched_clock() in psi")
    Cc: stable@vger.kernel.org
    Signed-off-by: default avatarJohannes Weiner <hannes@cmpxchg.org>
    Reviewed-by: default avatarChengming Zhou <chengming.zhou@linux.dev>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    3840cbe2
psi.c 46.2 KB