Commit 75f80859 authored by Peter Zijlstra's avatar Peter Zijlstra Committed by Ingo Molnar

perf/x86/intel/pebs: Robustify PEBS buffer drain

Vince Weaver and Stephane Eranian reported warnings in the PEBS
code when running the perf fuzzer. Stephane wrote:

  > I can reproduce the problem on my HSW running the fuzzer.
  >
  > I can see why this could be happening if you are mixing PEBS and non PEBS events
  > in the bottom 4 counters. I suspect:
  >         for (bit = 0; bit < x86_pmu.max_pebs_events; bit++) {
  >                 if ((counts[bit] == 0) && (error[bit] == 0))
  >                         continue;
  >
  > This test is not correct when you have non-PEBS events mixed with
  > PEBS events and they overflow at the same time. They will have
  > counts[i] != 0 but error[i] == 0, and thus you fall thru the loop
  > and hit the assert. Or it is something along those lines.

The only way I can make this work is if ->status only has !PEBS events
set, because if it has both set we'll take that slow path which masks
out the !PEBS bits.

After masking there are 3 options:

 - there is one bit set, and its @bit, we increment counts[bit].

 - there are multiple bits set, we increment error[] for each set bit,
   we do not increment counts[].

 - there are no bits set, we do nothing.

The intent was to never increment counts[] for !PEBS events.

Now if we start out with only a single !PEBS event set, we'll pass the
test and increment counts[] for a !PEBS and hit the warn.
Reported-by: default avatarVince Weaver <vincent.weaver@maine.edu>
Reported-by: default avatarStephane Eranian <eranian@google.com>
Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: kan.liang@intel.com
Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
parent 2a853e11
...@@ -1188,6 +1188,7 @@ static void intel_pmu_drain_pebs_nhm(struct pt_regs *iregs) ...@@ -1188,6 +1188,7 @@ static void intel_pmu_drain_pebs_nhm(struct pt_regs *iregs)
for (at = base; at < top; at += x86_pmu.pebs_record_size) { for (at = base; at < top; at += x86_pmu.pebs_record_size) {
struct pebs_record_nhm *p = at; struct pebs_record_nhm *p = at;
u64 pebs_status;
/* PEBS v3 has accurate status bits */ /* PEBS v3 has accurate status bits */
if (x86_pmu.intel_cap.pebs_format >= 3) { if (x86_pmu.intel_cap.pebs_format >= 3) {
...@@ -1198,12 +1199,17 @@ static void intel_pmu_drain_pebs_nhm(struct pt_regs *iregs) ...@@ -1198,12 +1199,17 @@ static void intel_pmu_drain_pebs_nhm(struct pt_regs *iregs)
continue; continue;
} }
bit = find_first_bit((unsigned long *)&p->status, pebs_status = p->status & cpuc->pebs_enabled;
pebs_status &= (1ULL << x86_pmu.max_pebs_events) - 1;
bit = find_first_bit((unsigned long *)&pebs_status,
x86_pmu.max_pebs_events); x86_pmu.max_pebs_events);
if (bit >= x86_pmu.max_pebs_events) if (WARN(bit >= x86_pmu.max_pebs_events,
continue; "PEBS record without PEBS event! status=%Lx pebs_enabled=%Lx active_mask=%Lx",
if (!test_bit(bit, cpuc->active_mask)) (unsigned long long)p->status, (unsigned long long)cpuc->pebs_enabled,
*(unsigned long long *)cpuc->active_mask))
continue; continue;
/* /*
* The PEBS hardware does not deal well with the situation * The PEBS hardware does not deal well with the situation
* when events happen near to each other and multiple bits * when events happen near to each other and multiple bits
...@@ -1218,27 +1224,21 @@ static void intel_pmu_drain_pebs_nhm(struct pt_regs *iregs) ...@@ -1218,27 +1224,21 @@ static void intel_pmu_drain_pebs_nhm(struct pt_regs *iregs)
* one, and it's not possible to reconstruct all events * one, and it's not possible to reconstruct all events
* that caused the PEBS record. It's called collision. * that caused the PEBS record. It's called collision.
* If collision happened, the record will be dropped. * If collision happened, the record will be dropped.
*
*/ */
if (p->status != (1 << bit)) { if (p->status != (1ULL << bit)) {
u64 pebs_status; for_each_set_bit(i, (unsigned long *)&pebs_status,
x86_pmu.max_pebs_events)
/* slow path */ error[i]++;
pebs_status = p->status & cpuc->pebs_enabled; continue;
pebs_status &= (1ULL << MAX_PEBS_EVENTS) - 1;
if (pebs_status != (1 << bit)) {
for_each_set_bit(i, (unsigned long *)&pebs_status,
MAX_PEBS_EVENTS)
error[i]++;
continue;
}
} }
counts[bit]++; counts[bit]++;
} }
for (bit = 0; bit < x86_pmu.max_pebs_events; bit++) { for (bit = 0; bit < x86_pmu.max_pebs_events; bit++) {
if ((counts[bit] == 0) && (error[bit] == 0)) if ((counts[bit] == 0) && (error[bit] == 0))
continue; continue;
event = cpuc->events[bit]; event = cpuc->events[bit];
WARN_ON_ONCE(!event); WARN_ON_ONCE(!event);
WARN_ON_ONCE(!event->attr.precise_ip); WARN_ON_ONCE(!event->attr.precise_ip);
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment