- 02 Sep, 2016 7 commits
-
-
Ard Biesheuvel authored
Now that the only remaining occurrences of the use of callee saved registers are on the primary boot path, add a comment to the code which register is used for what. Reviewed-by:
Mark Rutland <mark.rutland@arm.com> Signed-off-by:
Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by:
Will Deacon <will.deacon@arm.com>
-
Ard Biesheuvel authored
Instead of stashing the value of the link register in x28 before setting up the stack and calling into C code, create an ordinary PCS compatible stack frame so that we can push the return address onto the stack. Since exception handlers require a stack as well, assign the stack pointer register before installing the vector table. Note that this accounts for the difference between THREAD_START_SP and THREAD_SIZE, given that the stack pointer is always decremented before calling into any C code. Reviewed-by:
-
Ard Biesheuvel authored
Keeping __PHYS_OFFSET in x24 is actually less clear than simply taking the value of __PHYS_OFFSET using an adrp instruction in the three places that we need it. So change that. Reviewed-by:
-
Ard Biesheuvel authored
Using x27 for passing to __enable_mmu what is essentially the return address makes the code look more complicated than it needs to be. So switch to x30/lr, and update the secondary and cpu_resume call sites to simply call __enable_mmu as an ordinary function, with a bl instruction. This requires the callers to be covered by .idmap.text. Reviewed-by:
-
Ard Biesheuvel authored
The KASLR processing is only used by the primary boot path, and complements the processing that takes place in __primary_switch(). Move the two parts together, to make the code easier to understand. Also, fix up a minor whitespace issue. Reviewed-by:
-
Ard Biesheuvel authored
The function el2_setup() passes its return value in register w20, and in the two cases where the caller actually cares about this return value, it is passed into set_cpu_boot_mode_flag() [almost] directly, which expects its input in w20 as well. So there is no reason to use a 'special' callee saved register here, but we can simply follow the PCS for return value and first argument, respectively. Reviewed-by:
-
Ard Biesheuvel authored
This fixes a number of style issues in sleep.S. No functional changes are intended: - replace absolute literal references with relative references in __cpu_suspend_enter(), which executes from its virtual address - replace explicit lr assignment plus branch with bl in cpu_resume(), which aligns it with stext() and secondary_startup() - don't export _cpu_resume() - use adr_l for mpidr_hash reference, and fix the incorrect accompanying comment, which has been out of date since commit cabe1c81 ("arm64: Change cpu_resume() to enable mmu early then access sleep_sp by va") - replace leading spaces with tabs, and add a bit of whitespace for readability Reviewed-by:
-
- 01 Sep, 2016 4 commits
-
-
Vladimir Murzin authored
Commit e19a6ee2 ("arm64: kernel: Save and restore UAO and addr_limit on exception entry") states that exception handler inherits the original PSTATE.UAO value, so UAO needes to be reset explicitly. However, ARM 8.2 Extension documentation says: PSTATE.UAO is copied to SPSR_ELx.UAO and is then set to 0 on an exception taken from AArch64 to AArch64 so hardware already does the right thing. Signed-off-by:
Vladimir Murzin <vladimir.murzin@arm.com> Acked-by:
James Morse <james.morse@arm.com> Signed-off-by:
-
Will Deacon authored
The arm64 debug monitor initialisation code uses a CPU hotplug notifier to clear the OS lock when CPUs come online. This patch converts the code to the new hotplug mechanism. Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Reviewed-by:
Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Signed-off-by:
-
Will Deacon authored
The arm64 hw_breakpoint implementation uses a CPU hotplug notifier to reset the {break,watch}point registers when CPUs come online. This patch converts the code to the new hotplug mechanism, whilst moving the invocation earlier to remove the need to disable IRQs explicitly in the driver (which could cause havok if we trip a watchpoint in an IRQ handler whilst restoring the debug register state). Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Reviewed-by: -
zijun_hu authored
remove duplicate macro __KERNEL__ check Signed-off-by:
zijun_hu <zijun_hu@htc.com> Signed-off-by:
-
- 31 Aug, 2016 7 commits
-
-
Will Deacon authored
When TIF_SINGLESTEP is set for a task, the single-step state machine is enabled and we must take care not to reset it to the active-not-pending state if it is already in the active-pending state. Unfortunately, that's exactly what user_enable_single_step does, by unconditionally setting the SS bit in the SPSR for the current task. This causes failures in the GDB testsuite, where GDB ends up missing expected step traps if the instruction being stepped generates another trap, e.g. PTRACE_EVENT_FORK from an SVC instruction. This patch fixes the problem by preserving the current state of the stepping state machine when TIF_SINGLESTEP is set on the current thread. Cc: <stable@vger.kernel.org> Reported-by:
Yao Qi <yao.qi@arm.com> Signed-off-by:
-
Ard Biesheuvel authored
Expose the arm64_ftr_reg struct covering CTR_EL0 outside of cpufeature.o so that other code can refer to it directly (i.e., without performing the binary search) Reviewed-by:
Suzuki K Poulose <suzuki.poulose@arm.com> Signed-off-by:
-
Ard Biesheuvel authored
Constify the arm64_ftr_regs array, by moving the mutable arm64_ftr_reg fields out of the array itself. This also streamlines the bsearch, since the entire array can be covered by fewer cachelines. Moving the payload out of the array also allows us to have special explicitly defined struct instance in case other code needs to refer to it directly. Note that this replaces the runtime sorting of the array with a runtime BUG() check whether the array is sorted correctly in the code. Reviewed-by:
-
Ard Biesheuvel authored
The arm64_ftr_bits structures are never modified, so make them read-only. Reviewed-by:
-
Kefeng Wang authored
The UDBG_UNDEFINED/SYSCALL/BADABORT/SEGV are only used to show verbose user fault messages in arm, not arm64, drop them. Signed-off-by:
Kefeng Wang <wangkefeng.wang@huawei.com> Signed-off-by:
-
Kim Phillips authored
Any arm64 based parts that have cache aliasing issues can set it manually. Apparently dragged in from ARM(32) defaults in commit 8c2c3df3 "arm64: Build infrastructure". Signed-off-by:
Kim Phillips <kim.phillips@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Signed-off-by:
-
Michal Marek authored
The make rpm target depends on proper UTS_MACHINE definition. Also, use the variable in arch/arm64/kernel/setup.c, so that it's not accidentally removed in the future. Reported-and-tested-by:
Fabian Vogt <fvogt@suse.com> Signed-off-by:
Michal Marek <mmarek@suse.com> Signed-off-by:
-
- 26 Aug, 2016 5 commits
-
-
Will Deacon authored
Cortex-A53 erratum 843419 is worked around by the linker, although it is a configure-time option to GCC as to whether ld is actually asked to apply the workaround or not. This patch ensures that we pass --fix-cortex-a53-843419 to the linker when both CONFIG_ARM64_ERRATUM_843419=y and the linker supports the option. Acked-by:
-
James Morse authored
Now that we use the MPIDR to resume on the same CPU that we hibernated on, we no longer need to refuse to hibernate if the boot cpu is offline. (Which we can't possibly know if kexec causes logical CPUs to be renumbered). This reverts commit 1fe492ce. Signed-off-by:
Catalin Marinas <catalin.marinas@arm.com> Signed-off-by:
-
James Morse authored
disable_nonboot_cpus() assumes that the lowest numbered online CPU is the boot CPU, and that this is the correct CPU to run any power management code on. On arm64 CPU0 can be taken offline. For hibernate/resume this means we may hibernate on a CPU other than CPU0. If the system is rebooted with kexec 'CPU0' will be assigned to a different CPU. This complicates hibernate/resume as now we can't trust the CPU numbers. We currently forbid hibernate if CPU0 has been hotplugged out to avoid this situation without kexec. Save the MPIDR of the CPU we hibernated on in the hibernate arch-header, use hibernate_resume_nonboot_cpu_disable() to direct which CPU we should resume on based on the MPIDR of the CPU we hibernated on. This allows us to hibernate/resume on any CPU, even if the logical numbers have been shuffled by kexec. Signed-off-by:
-
James Morse authored
disable_nonboot_cpus() assumes that the lowest numbered online CPU is the boot CPU, and that this is the correct CPU to run any power management code on. On x86 this is always correct, as CPU0 cannot (easily) by taken offline. On arm64 CPU0 can be taken offline. For hibernate/resume this means we may hibernate on a CPU other than CPU0. If the system is rebooted with kexec 'CPU0' will be assigned to a different physical CPU. This complicates hibernate/resume as now we can't trust the CPU numbers. Arch code can find the correct physical CPU, and ensure it is online before resume from hibernate begins, but also needs to influence disable_nonboot_cpus()s choice of CPU. Rename disable_nonboot_cpus() as freeze_secondary_cpus() and add an argument indicating which CPU should be left standing. Follow the logic in migrate_to_reboot_cpu() to use the lowest numbered online CPU if the requested CPU is not online. Add disable_nonboot_cpus() as an inline function that has the existing behaviour. Cc: Rafael J. Wysocki <rjw@rjwysocki.net> Reviewed-by:
Thomas Gleixner <tglx@linutronix.de> Signed-off-by:
-
Mark Rutland authored
Follow the example set by x86 in commit 9ccaf77c ("x86/mm: Always enable CONFIG_DEBUG_RODATA and remove the Kconfig option"), and make these protections a fundamental security feature rather than an opt-in. This also results in a minor code simplification. For those rare cases when users wish to disable this protection (e.g. for debugging), this can be done by passing 'rodata=off' on the command line. As DEBUG_RODATA_ALIGN is only intended to address a performance/memory tradeoff, and does not affect correctness, this is left user-selectable. DEBUG_MODULE_RONX is also left user-selectable until the core code provides a boot-time option to disable the protection for debugging use-cases. Cc: Catalin Marinas <catalin.marinas@arm.com> Acked-by:
Kees Cook <keescook@chromium.org> Acked-by:
Laura Abbott <labbott@redhat.com> Signed-off-by:
-
- 25 Aug, 2016 6 commits
-
-
AKASHI Takahiro authored
Kdump(kexec-tools) parses /proc/iomem to identify all the memory regions on the system. Since the current kernel names "nomap" regions, like UEFI runtime services code/data, as "System RAM," kexec-tools sets up elf core header to include them in a crash dump file (/proc/vmcore). Then crash dump kernel parses UEFI memory map again, re-marks those regions as "nomap" and does not create a memory mapping for them unlike the other areas of System RAM. In this case, copying /proc/vmcore through copy_oldmem_page() on crash dump kernel will end up with a kernel abort, as reported in [1]. This patch names all the "nomap" regions explicitly as "reserved" so that we can exclude them from a crash dump file. acpi_os_ioremap() must also be modified because those regions have WB attributes [2]. Apart from kdump, this change also matches x86's use of acpi (and /proc/iomem). [1] http://lists.infradead.org/pipermail/linux-arm-kernel/2016-August/448186.html [2] http://lists.infradead.org/pipermail/linux-arm-kernel/2016-August/450089.htmlReviewed-by:
AKASHI Takahiro <takahiro.akashi@linaro.org> Signed-off-by:
-
James Morse authored
DEBUG_PAGEALLOC removes the valid bit of page table entries to prevent any access to unallocated memory. Hibernate uses this as a hint that those pages don't need to be saved/restored. This patch adds the kernel_page_present() function it uses. hibernate.c copies the resume kernel's linear map for use during restore. Add _copy_pte() to fill-in the holes made by DEBUG_PAGEALLOC in the resume kernel, so we can restore data the original kernel had at these addresses. Finally, DEBUG_PAGEALLOC means the linear-map alias of KERNEL_START to KERNEL_END may have holes in it, so we can't lazily clean this whole area to the PoC. Only clean the new mmuoff region, and the kernel/kvm idmaps. This reverts commit da24eb1f. Reported-by:
-
James Morse authored
Resume from hibernate needs to clean any text executed by the kernel with the MMU off to the PoC. Collect these functions together into the .idmap.text section as all this code is tightly coupled and also needs the same cleaning after resume. Data is more complicated, secondary_holding_pen_release is written with the MMU on, clean and invalidated, then read with the MMU off. In contrast __boot_cpu_mode is written with the MMU off, the corresponding cache line is invalidated, so when we read it with the MMU on we don't get stale data. These cache maintenance operations conflict with each other if the values are within a Cache Writeback Granule (CWG) of each other. Collect the data into two sections .mmuoff.data.read and .mmuoff.data.write, the linker script ensures mmuoff.data.write section is aligned to the architectural maximum CWG of 2KB. Signed-off-by:
-
James Morse authored
Each time new section markers are added, kernel/vmlinux.ld.S is updated, and new extern char __start_foo[] definitions are scattered through the tree. Create asm/include/sections.h to collect these definitions (and include the existing asm-generic version). Signed-off-by:
-
Catalin Marinas authored
The ARMv8 architecture allows execute-only user permissions by clearing the PTE_UXN and PTE_USER bits. However, the kernel running on a CPU implementation without User Access Override (ARMv8.2 onwards) can still access such page, so execute-only page permission does not protect against read(2)/write(2) etc. accesses. Systems requiring such protection must enable features like SECCOMP. This patch changes the arm64 __P100 and __S100 protection_map[] macros to the new __PAGE_EXECONLY attributes. A side effect is that pte_user() no longer triggers for __PAGE_EXECONLY since PTE_USER isn't set. To work around this, the check is done on the PTE_NG bit via the pte_ng() macro. VM_READ is also checked now for page faults. Reviewed-by:
-
Pratyush Anand authored
Whenever we are hitting a kprobe from a none-kprobe debug exception handler, we hit an infinite occurrences of "Unexpected kernel single-step exception at EL1" PSTATE.D is debug exception mask bit. It is set whenever we enter into an exception mode. When it is set then Watchpoint, Breakpoint, and Software Step exceptions are masked. However, software Breakpoint Instruction exceptions can never be masked. Therefore, if we ever execute a BRK instruction, irrespective of D-bit setting, we will be receiving a corresponding breakpoint exception. For example: - We are executing kprobe pre/post handler, and kprobe has been inserted in one of the instruction of a function called by handler. So, it executes BRK instruction and we land into the case of KPROBE_REENTER. (This case is already handled by current code) - We are executing uprobe handler or any other BRK handler such as in WARN_ON (BRK BUG_BRK_IMM), and we trace that path using kprobe.So, we enter into kprobe breakpoint handler,from another BRK handler.(This case is not being handled currently) In all such cases kprobe breakpoint exception will be raised when we were already in debug exception mode. SPSR's D bit (bit 9) shows the value of PSTATE.D immediately before the exception was taken. So, in above example cases we would find it set in kprobe breakpoint handler. Single step exception will always be followed by a kprobe breakpoint exception.However, it will only be raised gracefully if we clear D bit while returning from breakpoint exception. If D bit is set then, it results into undefined exception and when it's handler enables dbg then single step exception is generated, however it will never be handled(because address does not match and therefore treated as unexpected). This patch clears D-flag unconditionally in setup_singlestep, so that we can always get single step exception correctly after returning from breakpoint exception. Additionally, it also removes D-flag set statement for KPROBE_REENTER return path, because debug exception for KPROBE_REENTER will always take place in a debug exception state. So, D-flag will already be set in this case. Acked-by:
Sandeepa Prabhu <sandeepa.s.prabhu@gmail.com> Acked-by:
Masami Hiramatsu <mhiramat@kernel.org> Signed-off-by:
Pratyush Anand <panand@redhat.com> Signed-off-by:
-
- 22 Aug, 2016 10 commits
-
-
Ard Biesheuvel authored
Currently, x25 and x26 hold the physical addresses of idmap_pg_dir and swapper_pg_dir, respectively, when running early boot code. But having registers with 'global' scope in files that contain different sections with different lifetimes, and that are called by different CPUs at different times is a bit messy, especially since stashing the values does not buy us anything in terms of code size or clarity. So simply replace each reference to x25 or x26 with an adrp instruction referring to idmap_pg_dir or swapper_pg_dir directly. Acked-by:
-
Jisheng Zhang authored
These objects are set during initialization, thereafter are read only. Previously I only want to mark vdso_pages, vdso_spec, vectors_page and cpu_ops as __read_mostly from performance point of view. Then inspired by Kees's patch[1] to apply more __ro_after_init for arm, I think it's better to mark them as __ro_after_init. What's more, I find some more objects are also read only after init. So apply __ro_after_init to all of them. This patch also removes global vdso_pagelist and tries to clean up vdso_spec[] assignment code. [1] http://www.spinics.net/lists/arm-kernel/msg523188.htmlAcked-by:
Jisheng Zhang <jszhang@marvell.com> Signed-off-by:
-
Jisheng Zhang authored
The vm_special_mapping spec which is used for aarch32 vectors page is never modified, so mark it as const. Acked-by:
-
Jisheng Zhang authored
It is not needed after booting, this patch moves the alloc_vectors_page function to the __init section. Acked-by:
-
Masahiro Yamada authored
HAVE_CLK is select'ed by CLKDEV_LOOKUP, which is select'ed by COMMON_CLK, which is select'ed by ARM64. No sub-architecture needs to select HAVE_CLK explicitly. Signed-off-by:
Masahiro Yamada <yamada.masahiro@socionext.com> Signed-off-by:
-
Mark Rutland authored
Even though perf_ops_bp was removed/renamed back in commit b0a873eb ("perf: Register PMU implementations"), as part of v2.6.37, its definition still lives on in some arch headers. This patch removes the vestigal definition from arm64. Signed-off-by:
-
Kefeng Wang authored
Use the builtin_platform_driver() to simplify code. Signed-off-by:
-
Kwangwoo Lee authored
__dma_* routines have been converted to use start and size instread of start and end addresses. The patch was origianlly for adding __clean_dcache_area_poc() which will be used in pmem driver to clean dcache to the PoC(Point of Coherency) in arch_wb_cache_pmem(). The functionality of __clean_dcache_area_poc() was equivalent to __dma_clean_range(). The difference was __dma_clean_range() uses the end address, but __clean_dcache_area_poc() uses the size to clean. Thus, __clean_dcache_area_poc() has been revised with a fallthrough function of __dma_clean_range() after the change that __dma_* routines use start and size instead of using start and end. As a consequence of using start and size, the name of __dma_* routines has also been altered following the terminology below: area: takes a start and size range: takes a start and end Reviewed-by:Robin Murphy <robin.murphy@arm.com> Signed-off-by:
Kwangwoo Lee <kwangwoo.lee@sk.com> Signed-off-by:
-
Chris Metcalf authored
Currently ret_fast_syscall, work_pending, and ret_to_user form an ad-hoc state machine that can be difficult to reason about due to duplicated code and a large number of branch targets. This patch factors the common logic out into the existing do_notify_resume function, converting the code to C in the process, making the code more legible. This patch tries to closely mirror the existing behaviour while using the usual C control flow primitives. As local_irq_{disable,enable} may be instrumented, we balance exception entry (where we will almost most likely enable IRQs) with a call to trace_hardirqs_on just before the return to userspace. Signed-off-by:Chris Metcalf <cmetcalf@mellanox.com> Signed-off-by:
-
Mark Rutland authored
In break_before_make_ttbr_switch we perform broadcast TLB maintenance for the inner shareable domain, and use a DSB ISH to complete this. However, at the point we execute this, secondary CPUs are either physically offline, or executing code outside of the kernel. Upon entering the kernel, secondary CPUs will invalidate their TLBs before enabling their MMUs. Thus we do not need to invalidate TLBs of other CPUs, and as with idmap_cpu_replace_ttbr1 we can reduce the scope of maintenance to the TLBs of the local CPU. This keeps our TLB maintenance code consistent, and is a minor optimisation. Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Acked-by:
-
- 21 Aug, 2016 1 commit
-
-
Linus Torvalds authored
-
