• Andrew Morton's avatar
    [PATCH] SMP iowait stats · 3dd63b7d
    Andrew Morton authored
    Patch from William Lee Irwin III <wli@holomorphy.com>
    
    Idle time accounting is disturbed by the iowait statistics, for several
    reasons:
    
    (1) iowait time is not subdivided among cpus.
            The only way the distinction between idle time subtracted from
            cpus (in order to be accounted as iowait) can be made is by
            summing counters for a total and dividing the individual tick
            counters by the proportions. Any tick type resolution which is
            not properly per-cpu breaks this, meaning that cpus which are
            entirely idle, when any iowait is present on the system, will
            have all idle ticks accounted to iowait instead of true idle time.
    
    (2) kstat_read_proc() misreports iowait time
            The idle tick counter is passed twice to the sprintf(), once
            in the idle tick position, and once in the iowait tick position.
    
    (3) performance enhancement
            The O(1) scheduler was very carefully constructed to perform
            accesses only to localized cachelines whenever possible. The
            global counter violates one of its core design principles,
            and the localization of "most" accesses is in greater harmony
            with its overall design and provides (at the very least) a
            qualitative performance improvement wrt. cache.
    
    The method of correcting this is simple: embed an atomic iowait counter
    in the runqueues, find the runqueue being manipulated in io_schedule(),
    increment its atomic counter prior to schedule(), and decrement it
    after returning from schedule(), which is guaranteed to be the same one,
    as the counter incremented is tracked as a variable local to the procedure.
    Then simply sum to obtain a global iowait statistic.
    
    (Atomicity is required as the post-wait decrement may occur on a different
    cpu from the one owning the counter.)
    
    io_schedule() and io_schedule_timeout() are moved to sched.c as they must
    access the runqueues, which are private to sched.c, and nr_iowait() is
    created in order to export the sum of all runqueues' nr_iowait().
    3dd63b7d
sched.c 53.6 KB