- 02 Oct, 2020 1 commit
-
-
Jann Horn authored
Building linux-next with JUMP_LABEL=n and KASAN=y, I got this objtool warning: arch/x86/lib/copy_mc.o: warning: objtool: copy_mc_to_user()+0x22: call to __kasan_check_read() with UACCESS enabled What happens here is that copy_mc_to_user() branches on a static key in a UACCESS region: __uaccess_begin(); if (static_branch_unlikely(©_mc_fragile_key)) ret = copy_mc_fragile(to, from, len); ret = copy_mc_generic(to, from, len); __uaccess_end(); and the !CONFIG_JUMP_LABEL version of static_branch_unlikely() uses static_key_enabled(), which uses static_key_count(), which uses atomic_read(), which calls instrument_atomic_read(), which uses kasan_check_read(), which is __kasan_check_read(). Let's permit these KASAN helpers in UACCESS regions - static keys should probably work under UACCESS, I think. PeterZ adds: It's not a matter of permitting, it's a matter of being safe and correct. In this case it is, because it's a thin wrapper around check_memory_region() which was already marked safe. check_memory_region() is correct because the only thing it ends up calling is kasa_report() and that is also marked safe because that is annotated with user_access_save/restore() before it does anything else. On top of that, all of KASAN is noinstr, so nothing in here will end up in tracing and/or call schedule() before the user_access_save(). Signed-off-by:
Jann Horn <jannh@google.com> Acked-by:
Peter Zijlstra (Intel) <peterz@infradead.org> Signed-off-by:
Josh Poimboeuf <jpoimboe@redhat.com>
-
- 21 Sep, 2020 2 commits
-
-
Ilie Halip authored
With CONFIG_UBSAN_TRAP enabled, the compiler may insert a trap instruction after a call to a noreturn function. In this case, objtool warns that the UD2 instruction is unreachable. This is a behavior seen with Clang, from the oldest version capable of building the mainline x64_64 kernel (9.0), to the latest experimental version (12.0). Objtool silences similar warnings (trap after dead end instructions), so so expand that check to include dead end functions. Cc: Nick Desaulniers <ndesaulniers@google.com> Cc: Rong Chen <rong.a.chen@intel.com> Cc: Marco Elver <elver@google.com> Cc: Philip Li <philip.li@intel.com> Cc: Borislav Petkov <bp@alien8.de> Cc: kasan-dev@googlegroups.com Cc: x86@kernel.org Cc: clang-built-linux@googlegroups.com BugLink: https://github.com/ClangBuiltLinux/linux/issues/1148 Link: https://lore.kernel.org/lkml/CAKwvOdmptEpi8fiOyWUo=AiZJiX+Z+VHJOM2buLPrWsMTwLnyw@mail.gmail.comSuggested-by:
Nick Desaulniers <ndesaulniers@google.com> Reviewed-by:
kbuild test robot <lkp@intel.com> Signed-off-by:
Ilie Halip <ilie.halip@gmail.com> Tested-by:
Sedat Dilek <sedat.dilek@gmail.com> Signed-off-by:
-
Julien Thierry authored
Relocation for a call destination could point to a symbol that has type STT_NOTYPE. Lookup such a symbol when no function is available. Signed-off-by:
Julien Thierry <jthierry@redhat.com> Signed-off-by:
-
- 18 Sep, 2020 2 commits
-
-
Julien Thierry authored
It is possible for alternative code to unconditionally jump out of the alternative region. In such a case, if a fake jump is added at the end of the alternative instructions, the fake jump will never be reached. Since the fake jump is just a mean to make sure code validation does not go beyond the set of alternatives, reaching it is not a requirement. Signed-off-by:
-
Julien Thierry authored
save_reg already checks that the register being saved does not already have a saved state. Remove redundant checks before processing a register storing operation. Signed-off-by:
-
- 10 Sep, 2020 10 commits
-
-
Julien Thierry authored
The set of registers that can be included in an unwind hint and their encoding will depend on the architecture. Have arch specific code to decode that register. Signed-off-by:
-
Julien Thierry authored
Unwind hints are useful to provide objtool with information about stack states in non-standard functions/code. While the type of information being provided might be very arch specific, the mechanism to provide the information can be useful for other architectures. Move the relevant unwint hint definitions for all architectures to see. [ jpoimboe: REGS_IRET -> REGS_PARTIAL ] Signed-off-by:
-
Julien Thierry authored
Header include/linux/objtool.h contains both C and assembly definition that are visible regardless of the file including them. Place definition under conditional __ASSEMBLY__. Reviewed-by:
Miroslav Benes <mbenes@suse.cz> Signed-off-by:
-
Julien Thierry authored
Header frame.h is getting more code annotations to help objtool analyze object files. Rename the file to objtool.h. [ jpoimboe: add objtool.h to MAINTAINERS ] Signed-off-by:
-
Raphael Gault authored
The way to identify jump tables and retrieve all the data necessary to handle the different execution branches is not the same on all architectures. In order to be able to add other architecture support, define an arch-dependent function to process jump-tables. Reviewed-by:
Raphael Gault <raphael.gault@arm.com> [J.T.: Move arm64 bits out of this patch, Have only one function to find the start of the jump table, for now assume that the jump table format will be the same as x86] Signed-off-by:
-
Julien Thierry authored
As pointed out by the comment in handle_group_alt(), support of relocation for instructions in an alternative group depends on whether arch specific kernel code handles it. So, let objtool arch specific code decide whether a relocation for the alternative section should be accepted. Reviewed-by:
-
Julien Thierry authored
Some alternatives associated with a specific feature need to be treated in a special way. Since the features and how to treat them vary from one architecture to another, move the special case handling to arch specific code. Reviewed-by:
-
Julien Thierry authored
Some macros are defined to describe the size and layout of structures exception_table_entry, jump_entry and alt_instr. These values can vary from one architecture to another. Have the values be defined by arch specific code. Suggested-by:
-
Julien Thierry authored
Do not take into account outdated headers unrelated to the build of the current architecture. [ jpoimboe: use $SRCARCH directly ] Reviewed-by:
-
Julien Thierry authored
In order to support multiple architectures and potentially different sets of headers to compare against their kernel equivalent, it is simpler to have all headers to check in a single list. Reviewed-by:
-
- 01 Sep, 2020 22 commits
-
-
Julien Thierry authored
Implementation of ORC requires some definitions that are currently provided by the target architecture headers. Do not depend on these definitions when the orc subcommand is not implemented. This avoid requiring arches with no orc implementation to provide dummy orc definitions. Signed-off-by:
-
Julien Thierry authored
Orc generation is only done for text sections, but some instructions can be found in non-text sections (e.g. .discard.text sections). Skip setting their orc sections since their whole sections will be skipped for orc generation. Reviewed-by:
-
Julien Thierry authored
Now that the objtool_file can be obtained outside of the check function, orc generation builtin no longer requires check to explicitly call its orc related functions. Signed-off-by:
-
Julien Thierry authored
Structure objtool_file can be used by different subcommands. In fact it already is, by check and orc. Provide a function that allows to initialize objtool_file, that builtin can call, without relying on check to do the correct setup for them and explicitly hand the objtool_file to them. Reviewed-by:
-
Peter Zijlstra authored
Replace many of the indirect calls with static_call(). The average PMI time, as measured by perf_sample_event_took()*: PRE: 3283.03 [ns] POST: 3145.12 [ns] Which is a ~138 [ns] win per PMI, or a ~4.2% decrease. [*] on an IVB-EP, using: 'perf record -a -e cycles -- make O=defconfig-build/ -j80' Signed-off-by:
Ingo Molnar <mingo@kernel.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lore.kernel.org/r/20200818135805.338001015@infradead.org
-
Steven Rostedt (VMware) authored
Currently the tracepoint site will iterate a vector and issue indirect calls to however many handlers are registered (ie. the vector is long). Using static_call() it is possible to optimize this for the common case of only having a single handler registered. In this case the static_call() can directly call this handler. Otherwise, if the vector is longer than 1, call a function that iterates the whole vector like the current code. [peterz: updated to new interface] Signed-off-by:
Steven Rostedt (VMware) <rostedt@goodmis.org> Signed-off-by:
-
Peter Zijlstra authored
In order to use static_call() to wire up x86_pmu, we need to initialize earlier, specifically before memory allocation works; copy some of the tricks from jump_label to enable this. Primarily we overload key->next to store a sites pointer when there are no modules, this avoids having to use kmalloc() to initialize the sites and allows us to run much earlier. Signed-off-by:
-
Peter Zijlstra authored
Verify the text we're about to change is as we expect it to be. Requested-by:
-
Peter Zijlstra authored
GCC can turn our static_call(name)(args...) into a tail call, in which case we get a JMP.d32 into the trampoline (which then does a further tail-call). Teach objtool to recognise and mark these in .static_call_sites and adjust the code patching to deal with this. Signed-off-by:
-
Peter Zijlstra authored
Extend the static_call infrastructure to optimize the following common pattern: if (func_ptr) func_ptr(args...) For the trampoline (which is in effect a tail-call), we patch the JMP.d32 into a RET, which then directly consumes the trampoline call. For the in-line sites we replace the CALL with a NOP5. NOTE: this is 'obviously' limited to functions with a 'void' return type. NOTE: DEFINE_STATIC_COND_CALL() only requires a typename, as opposed to a full function. Signed-off-by: -
Peter Zijlstra authored
Future patches will need to poke a RET instruction, provide the infrastructure required for this. Signed-off-by:
-
Peter Zijlstra authored
Signed-off-by:
-
Josh Poimboeuf authored
Add the inline static call implementation for x86-64. The generated code is identical to the out-of-line case, except we move the trampoline into it's own section. Objtool uses the trampoline naming convention to detect all the call sites. It then annotates those call sites in the .static_call_sites section. During boot (and module init), the call sites are patched to call directly into the destination function. The temporary trampoline is then no longer used. [peterz: merged trampolines, put trampoline in section] Signed-off-by:
-
Josh Poimboeuf authored
Add the x86 out-of-line static call implementation. For each key, a permanent trampoline is created which is the destination for all static calls for the given key. The trampoline has a direct jump which gets patched by static_call_update() when the destination function changes. [peterz: fixed trampoline, rewrote patching code] Signed-off-by:
-
Peter Zijlstra authored
Similar to how we disallow kprobes on any other dynamic text (ftrace/jump_label) also disallow kprobes on inline static_call()s. Signed-off-by:
-
Josh Poimboeuf authored
Add infrastructure for an arch-specific CONFIG_HAVE_STATIC_CALL_INLINE option, which is a faster version of CONFIG_HAVE_STATIC_CALL. At runtime, the static call sites are patched directly, rather than using the out-of-line trampolines. Compared to out-of-line static calls, the performance benefits are more modest, but still measurable. Steven Rostedt did some tracepoint measurements: https://lkml.kernel.org/r/20181126155405.72b4f718@gandalf.local.home This code is heavily inspired by the jump label code (aka "static jumps"), as some of the concepts are very similar. For more details, see the comments in include/linux/static_call.h. [peterz: simplified interface; merged trampolines] Signed-off-by:
-
Josh Poimboeuf authored
Static calls are a replacement for global function pointers. They use code patching to allow direct calls to be used instead of indirect calls. They give the flexibility of function pointers, but with improved performance. This is especially important for cases where retpolines would otherwise be used, as retpolines can significantly impact performance. The concept and code are an extension of previous work done by Ard Biesheuvel and Steven Rostedt: https://lkml.kernel.org/r/20181005081333.15018-1-ard.biesheuvel@linaro.org https://lkml.kernel.org/r/20181006015110.653946300@goodmis.org There are two implementations, depending on arch support: 1) out-of-line: patched trampolines (CONFIG_HAVE_STATIC_CALL) 2) basic function pointers For more details, see the comments in include/linux/static_call.h. [peterz: simplified interface] Signed-off-by:
-
Josh Poimboeuf authored
The __ADDRESSABLE() macro uses the __LINE__ macro to create a temporary symbol which has a unique name. However, if the macro is used multiple times from within another macro, the line number will always be the same, resulting in duplicate symbols. Make the temporary symbols truly unique by using __UNIQUE_ID instead of __LINE__. Signed-off-by:
Ard Biesheuvel <ard.biesheuvel@linaro.org> Link: https://lore.kernel.org/r/20200818135804.564436253@infradead.org
-
Peter Zijlstra authored
Nothing ensures the module exists while we're iterating mod->jump_entries in __jump_label_mod_text_reserved(), take a module reference to ensure the module sticks around. Signed-off-by:
-
Peter Zijlstra authored
Now that notifiers got unbroken; use the proper interface to handle notifier errors and propagate them. There were already MODULE_STATE_COMING notifiers that failed; notably: - jump_label_module_notifier() - tracepoint_module_notify() - bpf_event_notify() By propagating this error, we fix those users. Signed-off-by:
Jessica Yu <jeyu@kernel.org> Acked-by:
-
Peter Zijlstra authored
While auditing all module notifiers I noticed a whole bunch of fail wrt the return value. Notifiers have a 'special' return semantics. As is; NOTIFY_DONE vs NOTIFY_OK is a bit vague; but notifier_from_errno(0) results in NOTIFY_OK and NOTIFY_DONE has a comment that says "Don't care". From this I've used NOTIFY_DONE when the function completely ignores the callback and notifier_to_error() isn't used. Signed-off-by:
Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Reviewed-by:
Joel Fernandes (Google) <joel@joelfernandes.org> Reviewed-by:
Robert Richter <rric@kernel.org> Acked-by:
-
Peter Zijlstra authored
The current notifiers have the following error handling pattern all over the place: int err, nr; err = __foo_notifier_call_chain(&chain, val_up, v, -1, &nr); if (err & NOTIFIER_STOP_MASK) __foo_notifier_call_chain(&chain, val_down, v, nr-1, NULL) And aside from the endless repetition thereof, it is broken. Consider blocking notifiers; both calls take and drop the rwsem, this means that the notifier list can change in between the two calls, making @nr meaningless. Fix this by replacing all the __foo_notifier_call_chain() functions with foo_notifier_call_chain_robust() that embeds the above pattern, but ensures it is inside a single lock region. Note: I switched atomic_notifier_call_chain_robust() to use the spinlock, since RCU cannot provide the guarantee required for the recovery. Note: software_resume() error handling was broken afaict. Signed-off-by:Rafael J. Wysocki <rafael.j.wysocki@intel.com> Link: https://lore.kernel.org/r/20200818135804.325626653@infradead.org
-
- 30 Aug, 2020 3 commits
-
-
Linus Torvalds authored
-
git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6Linus Torvalds authored
Pull crypto fixes from Herbert Xu: - fix regression in af_alg that affects iwd - restore polling delay in qat - fix double free in ingenic on error path - fix potential build failure in sa2ul due to missing Kconfig dependency * 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6: crypto: af_alg - Work around empty control messages without MSG_MORE crypto: sa2ul - add Kconfig selects to fix build error crypto: ingenic - Drop kfree for memory allocated with devm_kzalloc crypto: qat - add delay before polling mailbox
-
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tipLinus Torvalds authored
Pull x86 fixes from Thomas Gleixner: "Three interrupt related fixes for X86: - Move disabling of the local APIC after invoking fixup_irqs() to ensure that interrupts which are incoming are noted in the IRR and not ignored. - Unbreak affinity setting. The rework of the entry code reused the regular exception entry code for device interrupts. The vector number is pushed into the errorcode slot on the stack which is then lifted into an argument and set to -1 because that's regs->orig_ax which is used in quite some places to check whether the entry came from a syscall. But it was overlooked that orig_ax is used in the affinity cleanup code to validate whether the interrupt has arrived on the new target. It turned out that this vector check is pointless because interrupts are never moved from one vector to another on the same CPU. That check is a historical leftover from the time where x86 supported multi-CPU affinities, but not longer needed with the now strict single CPU affinity. Famous last words ... - Add a missing check for an empty cpumask into the matrix allocator. The affinity change added a warning to catch the case where an interrupt is moved on the same CPU to a different vector. This triggers because a condition with an empty cpumask returns an assignment from the allocator as the allocator uses for_each_cpu() without checking the cpumask for being empty. The historical inconsistent for_each_cpu() behaviour of ignoring the cpumask and unconditionally claiming that CPU0 is in the mask struck again. Sigh. plus a new entry into the MAINTAINER file for the HPE/UV platform" * tag 'x86-urgent-2020-08-30' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: genirq/matrix: Deal with the sillyness of for_each_cpu() on UP x86/irq: Unbreak interrupt affinity setting x86/hotplug: Silence APIC only after all interrupts are migrated MAINTAINERS: Add entry for HPE Superdome Flex (UV) maintainers
-
