- 09 Apr, 2017 40 commits
-
-
Marc Zyngier authored
In order to restore HYP mode to its original condition, KVM currently implements __kvm_hyp_reset(). As we're moving towards a hyp-stub defined API, it becomes necessary to implement HVC_RESET_VECTORS. This patch adds the HVC_RESET_VECTORS hypercall to the KVM init code, which so far lacked any form of hypercall support. Tested-by:
Keerthy <j-keerthy@ti.com> Acked-by:
Russell King <rmk+kernel@armlinux.org.uk> Acked-by:
Catalin Marinas <catalin.marinas@arm.com> Signed-off-by:
Marc Zyngier <marc.zyngier@arm.com> Signed-off-by:
Christoffer Dall <cdall@linaro.org>
-
Marc Zyngier authored
Let's define a new stub hypercall that resets the HYP configuration to its default: hyp-stub vectors, and MMU disabled. Of course, for the hyp-stub itself, this is a trivial no-op. Hypervisors will have a bit more work to do. Tested-by:
-
Marc Zyngier authored
Define a standard return value to be returned when a hyp stub call fails. Signed-off-by:
-
Marc Zyngier authored
The KVM code needs to be able to compute the address of symbols in its idmap page (the equivalent of a virt_to_idmap() call). Unfortunately, virt_to_idmap is slightly complicated, depending on the use of arch_phys_to_idmap_offset or not, and none of that is readily available at HYP. Instead, expose a single kimage_voffset variable which contains the offset between a kernel VA and its idmap address, enabling the VA->IDMAP conversion. This allows the KVM code to behave similarily to its arm64 counterpart. Tested-by:
-
Marc Zyngier authored
It is not really obvious why the restart address should be in r3 when communicated to the hyp-stub. r1 should be perfectly adequate, and consistent with the rest of the code. Tested-by:
-
Marc Zyngier authored
cpu_v7_reset() now takes a second parameter indicating whether we should reboot in HYP or not. Update the documentation to reflect this. Tested-by:
-
Marc Zyngier authored
The conversion of the HYP stub ABI to something similar to arm64 left the KVM code broken, as it doesn't know about the new stub numbering. Let's move the various #defines to virt.h, and let KVM use HVC_GET_VECTORS. Tested-by:
-
Russell King authored
When we soft-reboot (eg, kexec) from one kernel into the next, we need to ensure that we enter the new kernel in the same processor mode as when we were entered, so that (eg) the new kernel can install its own hypervisor - the old kernel's hypervisor will have been overwritten. In order to do this, we need to pass a flag to cpu_reset() so it knows what to do, and we need to modify the kernel's own hypervisor stub to allow it to handle a soft-reboot. As we are always guaranteed to install our own hypervisor if we're entered in HYP32 mode, and KVM will have moved itself out of the way on kexec/normal reboot, we can assume that our hypervisor is in place when we want to kexec, so changing our hypervisor API should not be a problem. Tested-by:
-
Russell King authored
Improve the hyp-stub ABI to allow it to do more than just get/set the vectors. We follow the example in ARM64, where r0 is used as an opcode with the other registers as an argument. Tested-by:
-
Marc Zyngier authored
Another missing stub hypercall is HVC_SOFT_RESTART. It turns out that it is pretty easy to implement in terms of HVC_RESET_VECTORS (since it needs to turn the MMU off). Acked-by:
-
Marc Zyngier authored
We are now able to use the hyp stub to reset HYP mode. Time to kiss __kvm_hyp_reset goodbye, and use __hyp_reset_vectors. Acked-by:
James Morse <james.morse@arm.com> Signed-off-by:
-
Marc Zyngier authored
We now have a full hyp-stub implementation in the KVM init code, but the main KVM code only supports HVC_GET_VECTORS, which is not enough. Instead of reinventing the wheel, let's reuse the init implementation by branching to the idmap page when called with a hyp-stub hypercall. Acked-by:
-
Marc Zyngier authored
Now that we have an infrastructure to handle hypercalls in the KVM init code, let's implement HVC_GET_VECTORS there. Acked-by:
-
Marc Zyngier authored
In order to restore HYP mode to its original condition, KVM currently implements __kvm_hyp_reset(). As we're moving towards a hyp-stub defined API, it becomes necessary to implement HVC_RESET_VECTORS. This patch adds the HVC_RESET_VECTORS hypercall to the KVM init code, which so far lacked any form of hypercall support. Acked-by:
-
Marc Zyngier authored
Let's define a new stub hypercall that resets the HYP configuration to its default: hyp-stub vectors, and MMU disabled. Of course, for the hyp-stub itself, this is a trivial no-op. Hypervisors will have a bit more work to do. Acked-by:
-
Marc Zyngier authored
Comments in asm/virt.h are slightly out of date, so let's align them with the new behaviour of the code. Signed-off-by:
-
Marc Zyngier authored
Define a standard return value to be returned when a hyp stub call fails, and make KVM use it for ARM_EXCEPTION_HYP_GONE (instead of using a KVM-specific value). Signed-off-by:
-
Marc Zyngier authored
The EL2 code is not corrupting lr anymore, so don't bother preserving it in the EL1 trampoline code. Acked-by:
-
Marc Zyngier authored
At the moment, we only save/restore lr if on VHE, as we rely only the EL1 code to have preserved it in the non-VHE case. As we're about to get rid of the latter, let's move the save/restore code to the do_el2_call macro, unifying both code paths. Acked-by:
-
Marc Zyngier authored
When entering the kernel hyp stub, we check whether or not we've made it here through an HVC instruction, clobbering lr (aka x30) in the process. This is completely pointless, as HVC is the only way to get here (all traps to EL2 are disabled, no interrupt override is applied). So let's remove this bit of code whose only point is to corrupt a valuable register. Acked-by:
-
Marc Zyngier authored
Instead of considering that a CP15 accessor has failed when returning false, let's consider that it is *always* successful (after all, we won't stand for an incomplete emulation). The return value now simply indicates whether we should skip the instruction (because it has now been emulated), or if we should leave the PC alone if the emulation has injected an exception. Reviewed-by:
-
Marc Zyngier authored
Reads from write-only system registers are generally confined to EL1 and not propagated to EL2 (that's what the architecture mantates). In order to be sure that we have a sane behaviour even in the unlikely event that we have a broken system, we still handle it in KVM. Same goes for write to RO registers. In that case, let's inject an undef into the guest. Reviewed-by:
-
Marc Zyngier authored
If we fail to emulate a mrrc instruction, we: 1) deliver an exception, 2) spit a nastygram on the console, 3) write back some garbage to Rt/Rt2 While 1) and 2) are perfectly acceptable, 3) is out of the scope of the architecture... Let's mimick the code in kvm_handle_cp_32 and be more cautious. Reviewed-by:
-
Marc Zyngier authored
Instead of considering that a sysreg accessor has failed when returning false, let's consider that it is *always* successful (after all, we won't stand for an incomplete emulation). The return value now simply indicates whether we should skip the instruction (because it has now been emulated), or if we should leave the PC alone if the emulation has injected an exception. Reviewed-by:
-
Marc Zyngier authored
PMSWINC_EL0 is a WO register, so let's UNDEF when reading from it (in the highly hypothetical case where this doesn't UNDEF at EL1). Reviewed-by:
-
Marc Zyngier authored
Reads from write-only system registers are generally confined to EL1 and not propagated to EL2 (that's what the architecture mantates). In order to be sure that we have a sane behaviour even in the unlikely event that we have a broken system, we still handle it in KVM. In that case, let's inject an undef into the guest. Let's also remove write_to_read_only which isn't used anywhere. Reviewed-by:
-
Marc Zyngier authored
access_pminten() and access_pmuserenr() can only be accessed when the CPU is in a priviledged mode. If it is not, let's inject an UNDEF exception. Reviewed-by:
-
Marc Zyngier authored
Both pmu_*_el0_disabled() and pmu_counter_idx_valid() perform checks on the validity of an access, but only return a boolean indicating if the access is valid or not. Let's allow these functions to also inject an UNDEF exception if the access was illegal. Reviewed-by:
-
Marc Zyngier authored
There is a lot of duplication in the pmu_*_el0_disabled helpers, and as we're going to modify them shortly, let's move all the common stuff in a single function. No functional change. Reviewed-by:
-
Christoffer Dall authored
There is no need to call any functions to fold LRs when we don't use any LRs and we don't need to mess with overflow flags, take spinlocks, or prune the AP list if the AP list is empty. Note: list_empty is a single atomic read (uses READ_ONCE) and can therefore check if a list is empty or not without the need to take the spinlock protecting the list. Reviewed-by:
-
Christoffer Dall authored
Now when we do an early init of the static parts of the VGIC data structures, we can do things like checking if the AP lists are empty directly without having to explicitly check if the vgic is initialized and reduce a bit of work in our critical path. Acked-by:
-
Christoffer Dall authored
Implement early initialization for both the distributor and the CPU interfaces. The basic idea is that even though the VGIC is not functional or not requested from user space, the critical path of the run loop can still call VGIC functions that just won't do anything, without them having to check additional initialization flags to ensure they don't look at uninitialized data structures. Acked-by:
-
Christoffer Dall authored
We don't use these fields anymore so let's nuke them completely. Acked-by:
Christoffer Dall <christoffer.dall@linaro.org>
-
Christoffer Dall authored
Now when we don't look at the MISR and EISR values anymore, we can get rid of the logic to save them in the GIC save/restore code. Acked-by:
-
Christoffer Dall authored
Since we always read back the LRs that we wrote to the guest and the MISR and EISR registers simply provide a summary of the configuration of the bits in the LRs, there is really no need to read back those status registers and process them. We might as well just signal the notifyfd when folding the LR state and save some cycles in the process. We now clear the underflow bit in the fold_lr_state functions as we only need to clear this bit if we had used all the LRs, so this is as good a place as any to do that work. Reviewed-by:
-
Christoffer Dall authored
We currently assume that all the interrupts in our AP list will be queued to LRs, but that's not necessarily the case, because some of them could have been migrated away to different VCPUs and only the VCPU thread itself can remove interrupts from its AP list. Therefore, slightly change the logic to only setting the underflow interrupt when we actually run out of LRs. As it turns out, this allows us to further simplify the handling in vgic_sync_hwstate in later patches. Acked-by:
-
Christoffer Dall authored
There is no need to calculate and maintain live_lrs when we always populate the lowest numbered LRs first on every entry and clear all LRs on every exit. Acked-by:
-
Shih-Wei Li authored
We do not need to flush vgic states in each world switch unless there is pending IRQ queued to the vgic's ap list. We can thus reduce the overhead by not grabbing the spinlock and not making the extra function call to vgic_flush_lr_state. Note: list_empty is a single atomic read (uses READ_ONCE) and can therefore check if a list is empty or not without the need to take the spinlock protecting the list. Reviewed-by:
Shih-Wei Li <shihwei@cs.columbia.edu> Signed-off-by:
-
Christoffer Dall authored
We don't have to save/restore the VMCR on every entry to/from the guest, since on GICv2 we can access the control interface from EL1 and on VHE systems with GICv3 we can access the control interface from KVM running in EL2. GICv3 systems without VHE becomes the rare case, which has to save/restore the register on each round trip. Note that userspace accesses may see out-of-date values if the VCPU is running while accessing the VGIC state via the KVM device API, but this is already the case and it is up to userspace to quiesce the CPUs before reading the CPU registers from the GIC for an up-to-date view. Reviewed-by:
Christoffer Dall <cdall@cs.columbia.edu> Signed-off-by:
-
Suzuki K Poulose authored
In order to perform an operation on a gpa range, we currently iterate over each page in a user memory slot for the given range. This is inefficient while dealing with a big range (e.g, a VMA), especially while unmaping a range. At present, with stage2 unmap on a range with a hugepage backed region, we clear the PMD when we unmap the first page in the loop. The remaining iterations simply traverse the page table down to the PMD level only to see that nothing is in there. This patch reworks the code to invoke the callback handlers on the biggest range possible within the memory slot to to reduce the number of times the handler is called. Cc: Marc Zyngier <marc.zyngier@arm.com> Reviewed-by:
Suzuki K Poulose <suzuki.poulose@arm.com> Signed-off-by:
-
