- 02 Apr, 2020 1 commit
-
-
Nicholas Piggin authored
Commit 3282a3da ("powerpc/64: Implement soft interrupt replay in C") broke the doorbell wakeup optimisation introduced by commit a9af97aa ("powerpc/64s: msgclr when handling doorbell exceptions from system reset"). This patch restores the msgclr, in C code. It's now done in the system reset wakeup path rather than doorbell interrupt replay where it used to be, because it is always the right thing to do in the wakeup case, but it may be rarely of use in other interrupt replay situations in which case it's wasted work - we would have to run measurements to see if that was a worthwhile optimisation, and I suspect it would not be. The results are similar to those in the original commit, test on POWER8 of context_switch selftests benchmark with polling idle disabled (e.g., always nap, giving cross-CPU IPIs) gives the following results: broken patched Different threads, same core: 317k/s 375k/s +18.7% Different cores: 280k/s 282k/s +1.0% Fixes: 3282a3da ("powerpc/64: Implement soft interrupt replay in C") Signed-off-by:
Nicholas Piggin <npiggin@gmail.com> Signed-off-by:
Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/20200402121212.1118218-1-npiggin@gmail.com
-
- 01 Apr, 2020 39 commits
-
-
Clement Courbet authored
Declaring setjmp()/longjmp() as taking longs makes the signature non-standard, and makes clang complain. In the past, this has been worked around by adding -ffreestanding to the compile flags. The implementation looks like it only ever propagates the value (in longjmp) or sets it to 1 (in setjmp), and we only call longjmp with integer parameters. This allows removing -ffreestanding from the compilation flags. Fixes: c9029ef9 ("powerpc: Avoid clang warnings around setjmp and longjmp") Cc: stable@vger.kernel.org # v4.14+ Signed-off-by:
Clement Courbet <courbet@google.com> Reviewed-by:
Nathan Chancellor <natechancellor@gmail.com> Tested-by:
-
Leonardo Bras authored
Before checking for cpu_type == NULL, this same copy happens, so doing it here will just write the same value to the t->oprofile_type again. Remove the repeated copy, as it is unnecessary. Signed-off-by:
Leonardo Bras <leonardo@linux.ibm.com> Reviewed-by:
Michael Neuling <mikey@neuling.org> Signed-off-by:
-
Naveen N. Rao authored
GCC v8 defaults to enabling -fasynchronous-unwind-tables due to https://gcc.gnu.org/r259298, which results in .eh_frame section being generated. This results in additional disk usage by the build, as well as the kernel modules. Since the kernel has no use for this, this section is discarded. Add -fno-asynchronous-unwind-tables to KBUILD_CFLAGS to suppress generation of .eh_frame section. Note that our VDSOs need .eh_frame, but are not affected by this change since our VDSO code are all in assembly. Reported-by:
Rasmus Villemoes <linux@rasmusvillemoes.dk> Signed-off-by:
Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com> Reviewed-by:
Segher Boessenkool <segher@kernel.crashing.org> Signed-off-by:
-
Naveen N. Rao authored
The original commit/discussion adding -fno-dwarf2-cfi-asm refers to R_PPC64_REL32 relocations not being handled by our module loader: http://lkml.kernel.org/r/20090224065112.GA6690@bombadil.infradead.org However, that is now handled thanks to commit 9f751b82 ("powerpc/module: Add support for R_PPC64_REL32 relocations"). So, drop this flag from our Makefile. Signed-off-by:
-
Mike Rapoport authored
The ISA_DMA_THRESHOLD variable is set by several platforms but never referenced. Remove it. Signed-off-by:
Mike Rapoport <rppt@linux.ibm.com> Reviewed-by:
Christoph Hellwig <hch@lst.de> Signed-off-by:
-
Shilpasri G Bhat authored
Commit bf957155 ("powerpc/powernv: Add support to clear sensor groups data") added a mechanism to clear sensor-group data via a sysfs interface. However, the ABI for that interface has not been documented. This patch documents the ABI for the sysfs interface for sensor-groups and clearing the sensor-groups. This patch was originally sent by Shilpasri G Bhat on the mailing list: https://lkml.org/lkml/2018/8/1/85Signed-off-by:
Shilpasri G Bhat <shilpa.bhat@linux.vnet.ibm.com> Signed-off-by:
Gautham R. Shenoy <ego@linux.vnet.ibm.com> Signed-off-by:
-
Michael Ellerman authored
We added a usage of try-run to pmu/ebb/Makefile to detect if the toolchain supported the -no-pie option. This fails if we build out-of-tree and the source tree is not writable, as try-run tries to write its temporary files to the current directory. That leads to the -no-pie option being silently dropped, which leads to broken executables with some toolchains. If we remove the redirect to /dev/null in try-run, we see the error: make[3]: Entering directory '/linux/tools/testing/selftests/powerpc/pmu/ebb' /usr/bin/ld: cannot open output file .54.tmp: Read-only file system collect2: error: ld returned 1 exit status make[3]: Nothing to be done for 'all'. And looking with strace we see it's trying to use a file that's in the source tree: lstat("/linux/tools/testing/selftests/powerpc/pmu/ebb/.54.tmp", 0x7ffffc0f83c8) We can fix it by setting TMPOUT to point to the $(OUTPUT) directory, and we can verify with strace it's now trying to write to the output directory: lstat("/output/kselftest/powerpc/pmu/ebb/.54.tmp", 0x7fffd1bf6bf8) And also see that the -no-pie option is now correctly detected. Fixes: 0695f8bc ("selftests/powerpc: Handle Makefile for unrecognized option") Cc: stable@vger.kernel.org # v5.5+ Signed-off-by: -
Michael Ellerman authored
Relocatable kernel builds produce a warning about .gnu.hash being an orphan section: ld: warning: orphan section `.gnu.hash' from `linker stubs' being placed in section `.gnu.hash' If we try to discard it the build fails: ld -EL -m elf64lppc -pie --orphan-handling=warn --build-id -o .tmp_vmlinux1 -T ./arch/powerpc/kernel/vmlinux.lds --whole-archive arch/powerpc/kernel/head_64.o arch/powerpc/kernel/entry_64.o ... sound/built-in.a net/built-in.a virt/built-in.a --no-whole-archive --start-group lib/lib.a --end-group ld: could not find section .gnu.hash So add an entry to explicitly retain it, as we do for .hash. Signed-off-by: -
Christophe Leroy authored
ptrace_triggered() is declared in asm/hw_breakpoint.h and only needed when CONFIG_HW_BREAKPOINT is set, so move it into hw_breakpoint.c Signed-off-by:
Christophe Leroy <christophe.leroy@c-s.fr> Signed-off-by:
-
Christophe Leroy authored
Create ippc_gethwdinfo() to handle PPC_PTRACE_GETHWDBGINFO and reduce ifdef mess Signed-off-by:
-
Christophe Leroy authored
Create ptrace_get_debugreg() to handle PTRACE_GET_DEBUGREG and reduce ifdef mess Signed-off-by:
-
Christophe Leroy authored
Move ADV_DEBUG_REGS functions out of ptrace.c, into ptrace-adv.c and ptrace-noadv.c Signed-off-by:
-
Christophe Leroy authored
Create a dedicated ptrace-view.c file. Signed-off-by:
-
Christophe Leroy authored
Move TRANSACTIONAL_MEM functions out of ptrace.c, into ptrace-tm.c Signed-off-by:
-
Christophe Leroy authored
Move CONFIG_SPE functions out of ptrace.c, into ptrace-spe.c Signed-off-by:
-
Christophe Leroy authored
Move CONFIG_ALTIVEC functions out of ptrace.c, into ptrace-altivec.c Signed-off-by:
-
Christophe Leroy authored
Move CONFIG_VSX functions out of ptrace.c, into ptrace-vsx.c and ptrace-novsx.c Signed-off-by:
-
Christophe Leroy authored
PARAMETER_SAVE_AREA_OFFSET is not used, drop it. Signed-off-by:
-
Christophe Leroy authored
Drop a bunch of #ifdefs CONFIG_PPC64 that are not vital. Signed-off-by:
-
Christophe Leroy authored
Remove unused header includes in ptrace.c and ptrace32.c Signed-off-by:
-
Christophe Leroy authored
In order to allow splitting of ptrace depending on the different CONFIG_ options, create a subdirectory dedicated to ptrace and move ptrace.c and ptrace32.c into it. Signed-off-by:
-
Nicholas Piggin authored
This reconciles interrupts in the system call case like all other interrupts. This allows system_call_common to be shared with the scv system call implementation in a subsequent patch. Signed-off-by:
-
Nicholas Piggin authored
Regular interrupt return restores NVGPRS whereas lite returns do not. This is clumsy: most interrupts can return without restoring NVGPRS in most of the time, but there are special cases that require it (when registers have been modified by the kernel). So change interrupt return to not restore NVGPRS, and have interrupt handlers restore them explicitly in the cases that requires it. Signed-off-by:
-
Nicholas Piggin authored
Implement the bulk of interrupt return logic in C. The asm return code must handle a few cases: restoring full GPRs, and emulating stack store. The stack store emulation is significantly simplfied, rather than creating a new return frame and switching to that before performing the store, it uses the PACA to keep a scratch register around to perform the store. The asm return code is moved into 64e for now. The new logic has made allowance for 64e, but I don't have a full environment that works well to test it, and even booting in emulated qemu is not great for stress testing. 64e shouldn't be too far off working with this, given a bit more testing and auditing of the logic. This is slightly faster on a POWER9 (page fault speed increases about 1.1%), probably due to reduced mtmsrd. mpe: Includes fixes from Nick for _TIF_EMULATE_STACK_STORE handling (including the fast_interrupt_return path), to remove trace_hardirqs_on(), and fixes the interrupt-return part of the MSR_VSX restore bug caught by tm-unavailable selftest. mpe: Incorporate fix from Nick: The return-to-kernel path has to replay any soft-pending interrupts if it is returning to a context that had interrupts soft-enabled. It has to do this carefully and avoid plain enabling interrupts if this is an irq context, which can cause multiple nesting of interrupts on the stack, and other unexpected issues. The code which avoided this case got the soft-mask state wrong, and marked interrupts as enabled before going around again to retry. This seems to be mostly harmless except when PREEMPT=y, this calls preempt_schedule_irq with irqs apparently enabled and runs into a BUG in kernel/sched/core.c Signed-off-by:
Michal Suchanek <msuchanek@suse.de> Signed-off-by:
-
Nicholas Piggin authored
When local_irq_enable() finds a pending soft-masked interrupt, it "replays" it by setting up registers like the initial interrupt entry, then calls into the low level handler to set up an interrupt stack frame and process the interrupt. This is not necessary, and uses more stack than needed. The high level interrupt handler can be called directly from C, with just pt_regs set up on stack. This should be faster and use less stack. Signed-off-by:
-
Nicholas Piggin authored
Kernel addresses and potentially other sensitive data could be leaked in volatile registers after a syscall. Signed-off-by:
-
Nicholas Piggin authored
System call entry and particularly exit code is beyond the limit of what is reasonable to implement in asm. This conversion moves all conditional branches out of the asm code, except for the case that all GPRs should be restored at exit. Null syscall test is about 5% faster after this patch, because the exit work is handled under local_irq_disable, and the hard mask and pending interrupt replay is handled after that, which avoids games with MSR. mpe: Includes subsequent fixes from Nick: This fixes 4 issues caught by TM selftests. First was a tm-syscall bug that hit due to tabort_syscall being called after interrupts were reconciled (in a subsequent patch), which led to interrupts being enabled before tabort_syscall was called. Rather than going through an un-reconciling interrupts for the return, I just go back to putting the test early in asm, the C-ification of that wasn't a big win anyway. Second is the syscall return _TIF_USER_WORK_MASK check would go into an infinite loop if _TIF_RESTORE_TM became set. The asm code uses _TIF_USER_WORK_MASK to brach to slowpath which includes restore_tm_state. Third is system call return was not calling restore_tm_state, I missed this completely (alhtough it's in the return from interrupt C conversion because when the asm syscall code encountered problems it would branch to the interrupt return code. Fourth is MSR_VEC missing from restore_math, which was caught by tm-unavailable selftest taking an unexpected facility unavailable interrupt when testing VSX unavailble exception with MSR.FP=1 MSR.VEC=1. Fourth case also has a fixup in a subsequent patch. Signed-off-by:
-
Nicholas Piggin authored
Signed-off-by:
-
Nicholas Piggin authored
powerpc has an optimisation where interrupts avoid saving the non-volatile (or callee saved) registers to the interrupt stack frame if they are not required. Two problems with this are that an interrupt does not always know whether it will need non-volatiles; and if it does need them, they can only be saved from the entry-scoped asm code (because we don't control what the C compiler does with these registers). system calls are the most difficult: some system calls always require all registers (e.g., fork, to copy regs into the child). Sometimes registers are only required under certain conditions (e.g., tracing, signal delivery). These cases require ugly logic in the call chains (e.g., ppc_fork), and require a lot of logic to be implemented in asm. So remove the optimisation for system calls, and always save NVGPRs on entry. Modern high performance CPUs are not so sensitive, because the stores are dense in cache and can be hidden by other expensive work in the syscall path -- the null syscall selftests benchmark on POWER9 is not slowed (124.40ns before and 123.64ns after, i.e., within the noise). Other interrupts retain the NVGPR optimisation for now. Signed-off-by:
-
Nicholas Piggin authored
The soft NMI handler does not reconcile interrupt state, so it should not return via the normal ret_from_except path. Return like other NMIs, using the EXCEPTION_RESTORE_REGS macro. This becomes important when the scv interrupt is implemented, which must handle soft-masked interrupts that have r13 set to something other than the PACA -- returning to kernel in this case must restore r13. Signed-off-by:
-
Nicholas Piggin authored
This adds IRQ_HARD_DIS to irq_happened. Although it doesn't seem to matter much because we're not allowed to enable irqs in an NMI handler, the soft-irq debugging code is becoming more strict about ensuring IRQ_HARD_DIS is in sync with MSR[EE], this may help avoid asserts or other issues. Add a comment explaining why MCE does not have this. Early machine check is generally much smaller and more contained code which will explode if you look at it wrong anyway as it runs in real mode, though there's an argument that we should do similar reconciling for the MCE as well. Signed-off-by:
-
Nicholas Piggin authored
Apart from SRESET, MCE, and syscall (hcall variant), the SRR type interrupts are not escalated to hypervisor mode, so are delivered to the OS. When running PR KVM, the OS is the hypervisor, and the guest runs with MSR[PR]=1 (ie. usermode), so these interrupts must test if a guest was running when interrupted. These tests are required at the real-mode entry points because the PR KVM host runs with LPCR[AIL]=0. In HV KVM and nested HV KVM, the guest always receives these interrupts, so there is no need for the host to make this test. So remove the tests if PR KVM is not configured. Signed-off-by:
-
Nicholas Piggin authored
A few of the non-standard handlers are left uncommented. Some more description could be added to some. Signed-off-by:
-
Nicholas Piggin authored
Remove more magic numbers and replace with nicely named bools. Signed-off-by:
-
Nicholas Piggin authored
The reduction in interrupt entry size allows some handlers to be re-inlined. Signed-off-by:
-
Nicholas Piggin authored
The hdec interrupt handler is reported to sometimes fire in Linux if KVM leaves it pending after a guest exists. This is harmless, so there is a no-op handler for it. The interrupt handler currently uses the regular kernel stack. Change this to avoid touching the stack entirely. This should be the last place where the regular Linux stack can be accessed with asynchronous interrupts (including PMI) soft-masked. It might be possible to take advantage of this invariant, e.g., to context switch the kernel stack SLB entry without clearing MSR[EE]. Signed-off-by:
-
Nicholas Piggin authored
Signed-off-by:
-
Nicholas Piggin authored
These are used infrequently enough they don't provide much help, so inline them. Signed-off-by:
-
Nicholas Piggin authored
Replace IEARLY=1 and IEARLY=2 with IBRANCH_COMMON, which controls if the entry code branches to a common handler; and IREALMODE_COMMON, which controls whether the common handler should remain in real mode. These special cases no longer avoid loading the SRR registers, there is no point as most of them load the registers immediately anyway. Signed-off-by:
-
