1. 06 Nov, 2019 27 commits
  2. 28 Oct, 2019 13 commits
    • Liang, Kan's avatar
      perf/core: Optimize perf_init_event() for TYPE_SOFTWARE · d44f821b
      Liang, Kan authored
      Andi reported that he was hitting the linear search in
      perf_init_event() a lot. Now that all !TYPE_SOFTWARE events should hit
      the IDR, make sure the TYPE_SOFTWARE events are at the head of the
      list such that we'll quickly find the right PMU (provided a valid
      event was given).
      Signed-off-by: default avatarLiang, Kan <kan.liang@linux.intel.com>
      Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
      Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Cc: Jiri Olsa <jolsa@redhat.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Mark Rutland <mark.rutland@arm.com>
      Cc: Namhyung Kim <namhyung@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Stephane Eranian <eranian@google.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Vince Weaver <vincent.weaver@maine.edu>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      d44f821b
    • Peter Zijlstra's avatar
      perf/core: Optimize perf_init_event() · 66d258c5
      Peter Zijlstra authored
      Andi reported that he was hitting the linear search in
      perf_init_event() a lot. Make more agressive use of the IDR lookup to
      avoid hitting the linear search.
      
      With exception of PERF_TYPE_SOFTWARE (which relies on a hideous hack),
      we can put everything in the IDR. On top of that, we can alias
      TYPE_HARDWARE and TYPE_HW_CACHE to TYPE_RAW on the lookup side.
      
      This greatly reduces the chances of hitting the linear search.
      Reported-by: default avatarAndi Kleen <andi@firstfloor.org>
      Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
      Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Cc: Jiri Olsa <jolsa@redhat.com>
      Cc: Kan <kan.liang@linux.intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Mark Rutland <mark.rutland@arm.com>
      Cc: Namhyung Kim <namhyung@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Stephane Eranian <eranian@google.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Vince Weaver <vincent.weaver@maine.edu>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      66d258c5
    • Peter Zijlstra's avatar
      perf/core: Optimize perf_install_in_event() · db0503e4
      Peter Zijlstra authored
      Andi reported that when creating a lot of events, a lot of time is
      spent in IPIs and asked if it would be possible to elide some of that.
      
      Now when, as for example the perf-tool always does, events are created
      disabled, then these events will not need to be scheduled when added
      to the context (they're still disable) and therefore the IPI is not
      required -- except for the very first event, that will need to set
      ctx->is_active.
      
      ( It might be possible to set ctx->is_active remotely for cpu_ctx, but
        we really need the IPI for task_ctx, so lets not make that
        distinction. )
      
      Also use __perf_effective_state() since group events depend on the
      state of the leader, if the leader is OFF, the whole group is OFF.
      
      So when sibling events are created enabled (XXX check tool) then we
      only need a single IPI to create and enable the whole group (+ that
      initial IPI to initialize the context).
      Suggested-by: default avatarAndi Kleen <andi@firstfloor.org>
      Reported-by: default avatarAndi Kleen <andi@firstfloor.org>
      Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
      Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Cc: Jiri Olsa <jolsa@redhat.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Mark Rutland <mark.rutland@arm.com>
      Cc: Namhyung Kim <namhyung@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Stephane Eranian <eranian@google.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Vince Weaver <vincent.weaver@maine.edu>
      Cc: acme@kernel.org
      Cc: kan.liang@linux.intel.com
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      db0503e4
    • Alexey Budankov's avatar
      perf/x86: Synchronize PMU task contexts on optimized context switches · c2b98a86
      Alexey Budankov authored
      Install Intel specific PMU task context synchronization adapter and
      extend optimized context switch path with PMU specific task context
      synchronization to fix LBR callstack virtualization on context switches.
      Signed-off-by: default avatarAlexey Budankov <alexey.budankov@linux.intel.com>
      Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
      Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
      Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Cc: Ian Rogers <irogers@google.com>
      Cc: Jiri Olsa <jolsa@redhat.com>
      Cc: Kan Liang <kan.liang@linux.intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Mark Rutland <mark.rutland@arm.com>
      Cc: Namhyung Kim <namhyung@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Song Liu <songliubraving@fb.com>
      Cc: Stephane Eranian <eranian@google.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Vince Weaver <vincent.weaver@maine.edu>
      Link: https://lkml.kernel.org/r/9c6445a9-bdba-ef03-3859-f1f91198f27a@linux.intel.comSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      c2b98a86
    • Alexey Budankov's avatar
      perf/x86/intel: Implement LBR callstack context synchronization · 421ca868
      Alexey Budankov authored
      Implement intel_pmu_lbr_swap_task_ctx() method updating counters
      of the events that requested LBR callstack data on a sample.
      
      The counter can be zero for the case when task context belongs to
      a thread that has just come from a block on a futex and the context
      contains saved (lbr_stack_state == LBR_VALID) LBR register values.
      
      For the values to be restored at LBR registers on the next thread's
      switch-in event it swaps the counter value with the one that is
      expected to be non zero at the previous equivalent task perf event
      context.
      
      Swap operation type ensures the previous task perf event context
      stays consistent with the amount of events that requested LBR
      callstack data on a sample.
      Signed-off-by: default avatarAlexey Budankov <alexey.budankov@linux.intel.com>
      Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
      Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
      Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Cc: Ian Rogers <irogers@google.com>
      Cc: Jiri Olsa <jolsa@redhat.com>
      Cc: Kan Liang <kan.liang@linux.intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Mark Rutland <mark.rutland@arm.com>
      Cc: Namhyung Kim <namhyung@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Song Liu <songliubraving@fb.com>
      Cc: Stephane Eranian <eranian@google.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Vince Weaver <vincent.weaver@maine.edu>
      Link: https://lkml.kernel.org/r/261ac742-9022-c3f4-5885-1eae7415b091@linux.intel.comSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      421ca868
    • Alexey Budankov's avatar
      perf/x86: Install platform specific ->swap_task_ctx() adapter · a4439970
      Alexey Budankov authored
      Bridge perf core and x86 swap_task_ctx() method calls.
      Signed-off-by: default avatarAlexey Budankov <alexey.budankov@linux.intel.com>
      Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
      Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
      Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Cc: Ian Rogers <irogers@google.com>
      Cc: Jiri Olsa <jolsa@redhat.com>
      Cc: Kan Liang <kan.liang@linux.intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Mark Rutland <mark.rutland@arm.com>
      Cc: Namhyung Kim <namhyung@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Song Liu <songliubraving@fb.com>
      Cc: Stephane Eranian <eranian@google.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Vince Weaver <vincent.weaver@maine.edu>
      Link: https://lkml.kernel.org/r/b157e97d-32c3-aeaf-13ba-47350c677906@linux.intel.comSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      a4439970
    • Alexey Budankov's avatar
      perf/core, perf/x86: Introduce swap_task_ctx() method at 'struct pmu' · fc1adfe3
      Alexey Budankov authored
      Declare swap_task_ctx() methods at the generic and x86 specific
      pmu types to bridge calls to platform specific PMU code on optimized
      context switch path between equivalent task perf event contexts.
      Signed-off-by: default avatarAlexey Budankov <alexey.budankov@linux.intel.com>
      Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
      Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
      Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Cc: Ian Rogers <irogers@google.com>
      Cc: Jiri Olsa <jolsa@redhat.com>
      Cc: Kan Liang <kan.liang@linux.intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Mark Rutland <mark.rutland@arm.com>
      Cc: Namhyung Kim <namhyung@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Song Liu <songliubraving@fb.com>
      Cc: Stephane Eranian <eranian@google.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Vince Weaver <vincent.weaver@maine.edu>
      Link: https://lkml.kernel.org/r/9a0aa84a-f062-9b64-3133-373658550c4b@linux.intel.comSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      fc1adfe3
    • Ingo Molnar's avatar
      65133033
    • Geert Uytterhoeven's avatar
      perf/headers: Fix spelling s/EACCESS/EACCES/, s/privilidge/privilege/ · 652521d4
      Geert Uytterhoeven authored
      As per POSIX, the correct spelling of the error code is EACCES:
      
        include/uapi/asm-generic/errno-base.h:#define EACCES 13 /* Permission denied */
      Signed-off-by: default avatarGeert Uytterhoeven <geert+renesas@glider.be>
      Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
      Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
      Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Cc: Jiri Kosina <trivial@kernel.org>
      Cc: Jiri Olsa <jolsa@redhat.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Mark Rutland <mark.rutland@arm.com>
      Cc: Namhyung Kim <namhyung@kernel.org>
      Cc: Stephane Eranian <eranian@google.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Vince Weaver <vincent.weaver@maine.edu>
      Link: https://lkml.kernel.org/r/20191024122904.12463-1-geert+renesas@glider.beSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      652521d4
    • Kan Liang's avatar
      perf/x86/uncore: Fix event group support · 75be6f70
      Kan Liang authored
      The events in the same group don't start or stop simultaneously.
      Here is the ftrace when enabling event group for uncore_iio_0:
      
        # perf stat -e "{uncore_iio_0/event=0x1/,uncore_iio_0/event=0xe/}"
      
                  <idle>-0     [000] d.h.  8959.064832: read_msr: a41, value
        b2b0b030		//Read counter reg of IIO unit0 counter0
                  <idle>-0     [000] d.h.  8959.064835: write_msr: a48, value
        400001			//Write Ctrl reg of IIO unit0 counter0 to enable
        counter0. <------ Although counter0 is enabled, Unit Ctrl is still
        freezed. Nothing will count. We are still good here.
                  <idle>-0     [000] d.h.  8959.064836: read_msr: a40, value
        30100                   //Read Unit Ctrl reg of IIO unit0
                  <idle>-0     [000] d.h.  8959.064838: write_msr: a40, value
        30000			//Write Unit Ctrl reg of IIO unit0 to enable all
        counters in the unit by clear Freeze bit  <------Unit0 is un-freezed.
        Counter0 has been enabled. Now it starts counting. But counter1 has not
        been enabled yet. The issue starts here.
                  <idle>-0     [000] d.h.  8959.064846: read_msr: a42, value 0
      			//Read counter reg of IIO unit0 counter1
                  <idle>-0     [000] d.h.  8959.064847: write_msr: a49, value
        40000e			//Write Ctrl reg of IIO unit0 counter1 to enable
        counter1.   <------ Now, counter1 just starts to count. Counter0 has
        been running for a while.
      
      Current code un-freezes the Unit Ctrl right after the first counter is
      enabled. The subsequent group events always loses some counter values.
      
      Implement pmu_enable and pmu_disable support for uncore, which can help
      to batch hardware accesses.
      
      No one uses uncore_enable_box and uncore_disable_box. Remove them.
      Signed-off-by: default avatarKan Liang <kan.liang@linux.intel.com>
      Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
      Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Cc: Jiri Olsa <jolsa@redhat.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Mark Rutland <mark.rutland@arm.com>
      Cc: Namhyung Kim <namhyung@kernel.org>
      Cc: Stephane Eranian <eranian@google.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Vince Weaver <vincent.weaver@maine.edu>
      Cc: linux-drivers-review@eclists.intel.com
      Cc: linux-perf@eclists.intel.com
      Fixes: 087bfbb0 ("perf/x86: Add generic Intel uncore PMU support")
      Link: https://lkml.kernel.org/r/1572014593-31591-1-git-send-email-kan.liang@linux.intel.comSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      75be6f70
    • Kim Phillips's avatar
      perf/x86/amd/ibs: Handle erratum #420 only on the affected CPU family (10h) · e431e79b
      Kim Phillips authored
      This saves us writing the IBS control MSR twice when disabling the
      event.
      
      I searched revision guides for all families since 10h, and did not
      find occurrence of erratum #420, nor anything remotely similar:
      so we isolate the secondary MSR write to family 10h only.
      
      Also unconditionally update the count mask for IBS Op implementations
      that have read & writeable current count (CurCnt) fields in addition
      to the MaxCnt field.  These bits were reserved on prior
      implementations, and therefore shouldn't have negative impact.
      Signed-off-by: default avatarKim Phillips <kim.phillips@amd.com>
      Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
      Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
      Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Jiri Olsa <jolsa@redhat.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Mark Rutland <mark.rutland@arm.com>
      Cc: Namhyung Kim <namhyung@kernel.org>
      Cc: Stephane Eranian <eranian@google.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Vince Weaver <vincent.weaver@maine.edu>
      Fixes: c9574fe0 ("perf/x86-ibs: Implement workaround for IBS erratum #420")
      Link: https://lkml.kernel.org/r/20191023150955.30292-2-kim.phillips@amd.comSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      e431e79b
    • Kim Phillips's avatar
      perf/x86/amd/ibs: Fix reading of the IBS OpData register and thus precise RIP validity · 317b96bb
      Kim Phillips authored
      The loop that reads all the IBS MSRs into *buf stopped one MSR short of
      reading the IbsOpData register, which contains the RipInvalid status bit.
      
      Fix the offset_max assignment so the MSR gets read, so the RIP invalid
      evaluation is based on what the IBS h/w output, instead of what was
      left in memory.
      Signed-off-by: default avatarKim Phillips <kim.phillips@amd.com>
      Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
      Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
      Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Jiri Olsa <jolsa@redhat.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Mark Rutland <mark.rutland@arm.com>
      Cc: Namhyung Kim <namhyung@kernel.org>
      Cc: Stephane Eranian <eranian@google.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Vince Weaver <vincent.weaver@maine.edu>
      Fixes: d47e8238 ("perf/x86-ibs: Take instruction pointer from ibs sample")
      Link: https://lkml.kernel.org/r/20191023150955.30292-1-kim.phillips@amd.comSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      317b96bb
    • Alexander Shishkin's avatar
      perf/core: Start rejecting the syscall with attr.__reserved_2 set · 8c7e9756
      Alexander Shishkin authored
      Commit:
      
        1a594131 ("perf: Add wakeup watermark control to the AUX area")
      
      added attr.__reserved_2 padding, but forgot to add an ABI check to reject
      attributes with this field set. Fix that.
      Signed-off-by: default avatarAlexander Shishkin <alexander.shishkin@linux.intel.com>
      Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Cc: Jiri Olsa <jolsa@redhat.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Mark Rutland <mark.rutland@arm.com>
      Cc: Namhyung Kim <namhyung@kernel.org>
      Cc: Stephane Eranian <eranian@google.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Vince Weaver <vincent.weaver@maine.edu>
      Cc: adrian.hunter@intel.com
      Cc: mathieu.poirier@linaro.org
      Link: https://lkml.kernel.org/r/20191025121636.75182-1-alexander.shishkin@linux.intel.comSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      8c7e9756