• Stephane Eranian's avatar
    perf: Fix broken interrupt rate throttling · e050e3f0
    Stephane Eranian authored
    This patch fixes the sampling interrupt throttling mechanism.
    
    It was broken in v3.2. Events were not being unthrottled. The
    unthrottling mechanism required that events be checked at each
    timer tick.
    
    This patch solves this problem and also separates:
    
      - unthrottling
      - multiplexing
      - frequency-mode period adjustments
    
    Not all of them need to be executed at each timer tick.
    
    This third version of the patch is based on my original patch +
    PeterZ proposal (https://lkml.org/lkml/2012/1/7/87).
    
    At each timer tick, for each context:
    
      - if the current CPU has throttled events, we unthrottle events
    
      - if context has frequency-based events, we adjust sampling periods
    
      - if we have reached the jiffies interval, we multiplex (rotate)
    
    We decoupled rotation (multiplexing) from frequency-mode sampling
    period adjustments.  They should not necessarily happen at the same
    rate. Multiplexing is subject to jiffies_interval (currently at 1
    but could be higher once the tunable is exposed via sysfs).
    
    We have grouped frequency-mode adjustment and unthrottling into the
    same routine to minimize code duplication. When throttled while in
    frequency mode, we scan the events only once.
    
    We have fixed the threshold enforcement code in __perf_event_overflow().
    There was a bug whereby it would allow more than the authorized rate
    because an increment of hwc->interrupts was not executed at the right
    place.
    
    The patch was tested with low sampling limit (2000) and fixed periods,
    frequency mode, overcommitted PMU.
    
    On a 2.1GHz AMD CPU:
    
     $ cat /proc/sys/kernel/perf_event_max_sample_rate
     2000
    
    We set a rate of 3000 samples/sec (2.1GHz/3000 = 700000):
    
     $ perf record -e cycles,cycles -c 700000  noploop 10
     $ perf report -D | tail -21
    
     Aggregated stats:
               TOTAL events:      80086
                MMAP events:         88
                COMM events:          2
                EXIT events:          4
            THROTTLE events:      19996
          UNTHROTTLE events:      19996
              SAMPLE events:      40000
    
     cycles stats:
               TOTAL events:      40006
                MMAP events:          5
                COMM events:          1
                EXIT events:          4
            THROTTLE events:       9998
          UNTHROTTLE events:       9998
              SAMPLE events:      20000
    
     cycles stats:
               TOTAL events:      39996
            THROTTLE events:       9998
          UNTHROTTLE events:       9998
              SAMPLE events:      20000
    
    For 10s, the cap is 2x2000x10 = 40000 samples.
    We get exactly that: 20000 samples/event.
    Signed-off-by: default avatarStephane Eranian <eranian@google.com>
    Cc: <stable@kernel.org> # v3.2+
    Signed-off-by: default avatarPeter Zijlstra <a.p.zijlstra@chello.nl>
    Link: http://lkml.kernel.org/r/20120126160319.GA5655@quadSigned-off-by: default avatarIngo Molnar <mingo@elte.hu>
    e050e3f0
core.c 161 KB