- 15 Mar, 2022 31 commits
-
-
Peter Zijlstra authored
These are code patching sites, not indirect targets. Signed-off-by:
Peter Zijlstra (Intel) <peterz@infradead.org> Acked-by:
Josh Poimboeuf <jpoimboe@redhat.com> Link: https://lore.kernel.org/r/20220308154318.936599479@infradead.org
-
Peter Zijlstra authored
Annotate away some of the generic code references. This is things where we take the address of a symbol for exception handling or return addresses (eg. context switch). Signed-off-by:
-
Peter Zijlstra authored
Assume firmware isn't IBT clean and disable it across calls. Signed-off-by:
Kees Cook <keescook@chromium.org> Acked-by:
-
Peter Zijlstra authored
Similar to ibt_selftest_ip, apply the same pattern. Suggested-by:
-
Peter Zijlstra authored
Signed-off-by:
-
Peter Zijlstra authored
The bits required to make the hardware go.. Of note is that, provided the syscall entry points are covered with ENDBR, #CP doesn't need to be an IST because we'll never hit the syscall gap. Signed-off-by:
-
Peter Zijlstra authored
Signed-off-by:
-
Peter Zijlstra authored
With IBT enabled builds we need ENDBR instructions at indirect jump target sites, since we start execution of the JIT'ed code through an indirect jump, the very first instruction needs to be ENDBR. Similarly, since eBPF tail-calls use indirect branches, their landing site needs to be an ENDBR too. The trampolines need similar adjustment. Signed-off-by:
Kumar Kartikeya Dwivedi <memxor@gmail.com> Acked-by:
-
Peter Zijlstra authored
In order to allow kprobes to skip the ENDBR instructions at sym+0 for X86_KERNEL_IBT builds, change _kprobe_addr() to take an architecture callback to inspect the function at hand and modify the offset if needed. This streamlines the existing interface to cover more cases and require less hooks. Once PowerPC gets fully converted there will only be the one arch hook. Signed-off-by:
Masami Hiramatsu <mhiramat@kernel.org> Acked-by:
-
Peter Zijlstra authored
Return trampoline must not use indirect branch to return; while this preserves the RSB, it is fundamentally incompatible with IBT. Instead use a retpoline like ROP gadget that defeats IBT while not unbalancing the RSB. And since ftrace_stub is no longer a plain RET, don't use it to copy from. Since RET is a trivial instruction, poke it directly. Signed-off-by:
-
Peter Zijlstra authored
Currently livepatch assumes __fentry__ lives at func+0, which is most likely untrue with IBT on. Instead make it use ftrace_location() by default which both validates and finds the actual ip if there is any in the same symbol. Suggested-by:
-
Peter Zijlstra authored
Currently a lot of ftrace code assumes __fentry__ is at sym+0. However with Intel IBT enabled the first instruction of a function will most likely be ENDBR. Change ftrace_location() to not only return the __fentry__ location when called for the __fentry__ location, but also when called for the sym+0 location. Then audit/update all callsites of this function to consistently use these new semantics. Suggested-by:
Steven Rostedt <rostedt@goodmis.org> Signed-off-by:
-
Peter Zijlstra authored
Signed-off-by:
-
Peter Zijlstra authored
The code does: ## branch into array mov jump_table(,%rax,8), %bufp JMP_NOSPEC bufp resulting in needing to mark the jump-table entries with ENDBR. Signed-off-by:
-
Peter Zijlstra authored
Signed-off-by:
-
Peter Zijlstra authored
Ensure the ASM functions have ENDBR on for IBT builds, this follows the ARM64 example. Unlike ARM64, we'll likely end up overwriting them with poison. Suggested-by:
Mark Rutland <mark.rutland@arm.com> Signed-off-by:
-
Peter Zijlstra authored
Kernel entry points should be having ENDBR on for IBT configs. The SYSCALL entry points are found through taking their respective address in order to program them in the MSRs, while the exception entry points are found through UNWIND_HINT_IRET_REGS. The rule is that any UNWIND_HINT_IRET_REGS at sym+0 should have an ENDBR, see the later objtool ibt validation patch. Signed-off-by:
-
Peter Zijlstra authored
Even though Xen currently doesn't advertise IBT, prepare for when it will eventually do so and sprinkle the ENDBR dust accordingly. Even though most of the entry points are IRET like, the CPL0 Hypervisor can set WAIT-FOR-ENDBR and demand ENDBR at these sites. Signed-off-by:
-
Peter Zijlstra authored
By doing an early rewrite of 'jmp native_iret` in restore_regs_and_return_to_kernel() we can get rid of the last INTERRUPT_RETURN user and paravirt_iret. Suggested-by:
Andrew Cooper <Andrew.Cooper3@citrix.com> Signed-off-by:
-
Peter Zijlstra authored
Since commit 5c8f6a2e ("x86/xen: Add xenpv_restore_regs_and_return_to_usermode()") Xen will no longer reach this code and we can do away with the paravirt SWAPGS/INTERRUPT_RETURN. Suggested-by:
-
Peter Zijlstra authored
Less duplication is more better. Signed-off-by:
-
Peter Zijlstra authored
Signed-off-by:
-
Peter Zijlstra authored
In order to have objtool warn about code references to !ENDBR instruction, we need an annotation to allow this for non-control-flow instances -- consider text range checks, text patching, or return trampolines etc. Signed-off-by:
-
Peter Zijlstra authored
Add Kconfig, Makefile and basic instruction support for x86 IBT. (Ab)use __DISABLE_EXPORTS to disable IBT since it's already employed to mark compressed and purgatory. Additionally mark realmode with it as well to avoid inserting ENDBR instructions there. While ENDBR is technically a NOP, inserting them was causing some grief due to code growth. There's also a problem with using __noendbr in code compiled without -fcf-protection=branch. Signed-off-by:
-
Peter Zijlstra authored
Currently WARN_FUNC() either prints func+off and failing that prints sec+off, add an intermediate sym+off. This is useful when playing around with entry code. Signed-off-by:
-
Peter Zijlstra authored
The current annotation relies on not running objtool on the file; this won't work when running objtool on vmlinux.o. Instead explicitly mark __efi64_thunk() to be ignored. This preserves the status quo, which is somewhat unfortunate. Luckily this code is hardly ever used. Signed-off-by:
-
Peter Zijlstra authored
Ignore all INT3 instructions for unreachable code warnings, similar to NOP. This allows using INT3 for various paddings instead of NOPs. Signed-off-by:
-
Peter Zijlstra authored
Add a --dry-run argument to skip writing the modifications. This is convenient for debugging. Signed-off-by:
Miroslav Benes <mbenes@suse.cz> Acked-by:
-
Peter Zijlstra authored
Without CONFIG_HAVE_STATIC_CALL_INLINE there's no point in creating the .static_call_sites section and it's related symbols. Signed-off-by:
-
Peter Zijlstra authored
Enjoy the cleanups and avoid conflicts vs linkage Signed-off-by: -
Fenghua Yu authored
The ENQCMD instruction implicitly accesses the PASID_MSR to fill in the pasid field of the descriptor being submitted to an accelerator. But there is no precise (and stable across kernel changes) point at which the PASID_MSR is updated from the value for one task to the next. Kernel code that uses accelerators must always use the ENQCMDS instruction which does not access the PASID_MSR. Check for use of the ENQCMD instruction in the kernel and warn on its usage. Signed-off-by:
Fenghua Yu <fenghua.yu@intel.com> Signed-off-by:
Borislav Petkov <bp@suse.de> Reviewed-by:
Tony Luck <tony.luck@intel.com> Acked-by:
-
- 13 Mar, 2022 2 commits
-
-
Linus Torvalds authored
-
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tipLinus Torvalds authored
Pull x86 fixes from Borislav Petkov: - Free shmem backing storage for SGX enclave pages when those are swapped back into EPC memory - Prevent do_int3() from being kprobed, to avoid recursion - Remap setup_data and setup_indirect structures properly when accessing their members - Correct the alternatives patching order for modules too * tag 'x86_urgent_for_v5.17_rc8' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: x86/sgx: Free backing memory after faulting the enclave page x86/traps: Mark do_int3() NOKPROBE_SYMBOL x86/boot: Add setup_indirect support in early_memremap_is_setup_data() x86/boot: Fix memremap of setup_indirect structures x86/module: Fix the paravirt vs alternative order
-
- 12 Mar, 2022 7 commits
-
-
Linus Torvalds authored
Merge tag 'perf-tools-fixes-for-v5.17-2022-03-12' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux Pull perf tools fixes from Arnaldo Carvalho de Melo: - Fix event parser error for hybrid systems - Fix NULL check against wrong variable in 'perf bench' and in the parsing code - Update arm64 KVM headers from the kernel sources - Sync cpufeatures header with the kernel sources * tag 'perf-tools-fixes-for-v5.17-2022-03-12' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux: perf parse: Fix event parser error for hybrid systems perf bench: Fix NULL check against wrong variable perf parse-events: Fix NULL check against wrong variable tools headers cpufeatures: Sync with the kernel sources tools kvm headers arm64: Update KVM headers from the kernel sources
-
git://anongit.freedesktop.org/drm/drmLinus Torvalds authored
Pull drm kconfig fix from Dave Airlie: "Thorsten pointed out this had fallen down the cracks and was in -next only, I've picked it out, fixed up it's Fixes: line. - fix regression in Kconfig" * tag 'drm-fixes-2022-03-12' of git://anongit.freedesktop.org/drm/drm: drm/panel: Select DRM_DP_HELPER for DRM_PANEL_EDP
-
Zhengjun Xing authored
This bug happened on hybrid systems when both cpu_core and cpu_atom have the same event name such as "UOPS_RETIRED.MS" while their event terms are different, then during perf stat, the event for cpu_atom will parse fail and then no output for cpu_atom. UOPS_RETIRED.MS -> cpu_core/period=0x1e8483,umask=0x4,event=0xc2,frontend=0x8/ UOPS_RETIRED.MS -> cpu_atom/period=0x1e8483,umask=0x1,event=0xc2/ It is because event terms in the "head" of parse_events_multi_pmu_add will be changed to event terms for cpu_core after parsing UOPS_RETIRED.MS for cpu_core, then when parsing the same event for cpu_atom, it still uses the event terms for cpu_core, but event terms for cpu_atom are different with cpu_core, the event parses for cpu_atom will fail. This patch fixes it, the event terms should be parsed from the original event. This patch can work for the hybrid systems that have the same event in more than 2 PMUs. It also can work in non-hybrid systems. Before: # perf stat -v -e UOPS_RETIRED.MS -a sleep 1 Using CPUID GenuineIntel-6-97-1 UOPS_RETIRED.MS -> cpu_core/period=0x1e8483,umask=0x4,event=0xc2,frontend=0x8/ Control descriptor is not initialized UOPS_RETIRED.MS: 2737845 16068518485 16068518485 Performance counter stats for 'system wide': 2,737,845 cpu_core/UOPS_RETIRED.MS/ 1.002553850 seconds time elapsed After: # perf stat -v -e UOPS_RETIRED.MS -a sleep 1 Using CPUID GenuineIntel-6-97-1 UOPS_RETIRED.MS -> cpu_core/period=0x1e8483,umask=0x4,event=0xc2,frontend=0x8/ UOPS_RETIRED.MS -> cpu_atom/period=0x1e8483,umask=0x1,event=0xc2/ Control descriptor is not initialized UOPS_RETIRED.MS: 1977555 16076950711 16076950711 UOPS_RETIRED.MS: 568684 8038694234 8038694234 Performance counter stats for 'system wide': 1,977,555 cpu_core/UOPS_RETIRED.MS/ 568,684 cpu_atom/UOPS_RETIRED.MS/ 1.004758259 seconds time elapsed Fixes: fb081153 ("perf parse-events: Allow config on kernel PMU events") Reviewed-by:Kan Liang <kan.liang@linux.intel.com> Signed-off-by:
Zhengjun Xing <zhengjun.xing@linux.intel.com> Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Alexander Shishkin <alexander.shishkin@intel.com> Cc: Andi Kleen <ak@linux.intel.com> Cc: Ian Rogers <irogers@google.com> Cc: Ingo Molnar <mingo@redhat.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Link: https://lore.kernel.org/r/20220307151627.30049-1-zhengjun.xing@linux.intel.comSigned-off-by:
Arnaldo Carvalho de Melo <acme@redhat.com>
-
Weiguo Li authored
We did a NULL check after "epollfdp = calloc(...)", but we checked "epollfd" instead of "epollfdp". Signed-off-by:
Weiguo Li <liwg06@foxmail.com> Acked-by:
Davidlohr Bueso <dave@stgolabs.net> Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com> Cc: Davidlohr Bueso <dbueso@suse.de> Cc: Ingo Molnar <mingo@redhat.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Link: https://lore.kernel.org/r/tencent_B5D64530EB9C7DBB8D2C88A0C790F1489D0A@qq.comSigned-off-by:
-
Weiguo Li authored
We did a null check after "tmp->symbol = strdup(...)", but we checked "list->symbol" other than "tmp->symbol". Reviewed-by:
John Garry <john.garry@huawei.com> Signed-off-by:
-
Arnaldo Carvalho de Melo authored
To pick the changes from: d45476d9 ("x86/speculation: Rename RETPOLINE_AMD to RETPOLINE_LFENCE") Its just a comment fixup. This only causes these perf files to be rebuilt: CC /tmp/build/perf/bench/mem-memcpy-x86-64-asm.o CC /tmp/build/perf/bench/mem-memset-x86-64-asm.o And addresses this perf build warning: Warning: Kernel ABI header at 'tools/arch/x86/include/asm/cpufeatures.h' differs from latest version at 'arch/x86/include/asm/cpufeatures.h' diff -u tools/arch/x86/include/asm/cpufeatures.h arch/x86/include/asm/cpufeatures.h Cc: Borislav Petkov <bp@suse.de> Cc: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lore.kernel.org/lkml/YiyiHatGaJQM7l/Y@kernel.orgSigned-off-by:
-
Arnaldo Carvalho de Melo authored
To pick the changes from: a5905d6a ("KVM: arm64: Allow SMCCC_ARCH_WORKAROUND_3 to be discovered and migrated") That don't causes any changes in tooling (when built on x86), only addresses this perf build warning: Warning: Kernel ABI header at 'tools/arch/arm64/include/uapi/asm/kvm.h' differs from latest version at 'arch/arm64/include/uapi/asm/kvm.h' diff -u tools/arch/arm64/include/uapi/asm/kvm.h arch/arm64/include/uapi/asm/kvm.h Cc: James Morse <james.morse@arm.com> Link: https://lore.kernel.org/lkml/YiyhAK6sVPc83FaI@kernel.orgSigned-off-by:
-
