- 28 Mar, 2017 40 commits
-
-
James Hogan authored
Properly implement emulation of the TLBR instruction for Trap & Emulate. This instruction reads the TLB entry pointed at by the CP0_Index register into the other TLB registers, which may have the side effect of changing the current ASID. Therefore abstract the CP0_EntryHi and ASID changing code into a common function in the process. A comment indicated that Linux doesn't use TLBR, which is true during normal use, however dumping of the TLB does use it (for example with the relatively recent 'x' magic sysrq key), as does a wired TLB entries test case in my KVM tests. Signed-off-by:
James Hogan <james.hogan@imgtec.com> Acked-by:
Ralf Baechle <ralf@linux-mips.org> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: "Radim Krčmář" <rkrcmar@redhat.com> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org
-
James Hogan authored
Octeon III has VZ ASE support, so allow KVM to be enabled on Octeon CPUs as it should now be functional. Signed-off-by: -
James Hogan authored
Octeon III implements a read-only guest CP0_PRid register, so add cases to the KVM register access API for Octeon to ensure the correct value is read and writes are ignored. Signed-off-by: -
James Hogan authored
Octeon III doesn't implement the optional GuestCtl0.CG bit to allow guest mode to execute virtual address based CACHE instructions, so implement emulation of a few important ones specifically for Octeon III in response to a GPSI exception. Currently the main reason to perform these operations is for icache synchronisation, so they are implemented as a simple icache flush with local_flush_icache_range(). Signed-off-by: -
James Hogan authored
Set up hardware virtualisation on Octeon III cores, configuring guest interrupt routing and carving out half of the root TLB for guest use, restoring it back again afterwards. We need to be careful to inhibit TLB shutdown machine check exceptions while invalidating guest TLB entries, since TLB invalidation is not available so guest entries must be invalidated by setting them to unique unmapped addresses, which could conflict with mappings set by the guest or root if recently repartitioned. Signed-off-by: -
James Hogan authored
Octeon CPUs don't report the correct dcache line size in CP0_Config1.DL, so encode the correct value for the guest CP0_Config1.DL based on cpu_dcache_line_size(). Signed-off-by: -
James Hogan authored
When TLB entries are invalidated in the presence of a virtually tagged icache, such as that found on Octeon CPUs, flush the icache so that we don't get a reserved instruction exception even though the TLB mapping is removed. Signed-off-by: -
James Hogan authored
Cache management is implemented separately for Cavium Octeon CPUs, so r4k_blast_[id]cache aren't available. Instead for Octeon perform a local icache flush using local_flush_icache_range(), and for other platforms which don't use c-r4k.c use __flush_cache_all() / flush_icache_all(). Signed-off-by: -
James Hogan authored
Add accessors for some VZ related Cavium Octeon III specific COP0 registers, along with field definitions. These will mostly be used by KVM to set up interrupt routing and partition the TLB between root and guest. Signed-off-by:
-
James Hogan authored
Create a trace event for guest mode changes, and enable VZ's GuestCtl0.MC bit after the trace event is enabled to trap all guest mode changes. The MC bit causes Guest Hardware Field Change (GHFC) exceptions whenever a guest mode change occurs (such as an exception entry or return from exception), so we need to handle this exception now. The MC bit is only enabled when restoring register state, so enabling the trace event won't take immediate effect. Tracing guest mode changes can be particularly handy when trying to work out what a guest OS gets up to before something goes wrong, especially if the problem occurs as a result of some previous guest userland exception which would otherwise be invisible in the trace. Signed-off-by: -
James Hogan authored
Transfer timer state to the VZ guest context (CP0_GTOffset & guest CP0_Count) when entering guest mode, enabling direct guest access to it, and transfer back to soft timer when saving guest register state. This usually allows guest code to directly read CP0_Count (via MFC0 and RDHWR) and read/write CP0_Compare, without trapping to the hypervisor for it to emulate the guest timer. Writing to CP0_Count or CP0_Cause.DC is much less common and still triggers a hypervisor GPSI exception, in which case the timer state is transferred back to an hrtimer before emulating the write. We are careful to prevent small amounts of drift from building up due to undeterministic time intervals between reading of the ktime and reading of CP0_Count. Some drift is expected however, since the system clocksource may use a different timer to the local CP0_Count timer used by VZ. This is permitted to prevent guest CP0_Count from appearing to go backwards. Signed-off-by: -
James Hogan authored
Add emulation of Memory Accessibility Attribute Registers (MAARs) when necessary. We can't actually do anything with whatever the guest provides, but it may not be possible to clear Guest.Config5.MRP so we have to emulate at least a pair of MAARs. Signed-off-by: -
James Hogan authored
When restoring guest state after another VCPU has run, be sure to clear CP0_LLAddr.LLB in order to break any interrupted atomic critical section. Without this SMP guest atomics don't work when LLB is present as one guest can complete the atomic section started by another guest. MIPS VZ guest read of CP0_LLAddr causes Guest Privileged Sensitive Instruction (GPSI) exception due to the address being root physical. Handle this by reporting only the LLB bit, which contains the bit for whether a ll/sc atomic is in progress without any reason for failure. Similarly on P5600 a guest write to CP0_LLAddr also causes a GPSI exception. Handle this also by clearing the guest LLB bit from root mode. Signed-off-by: -
James Hogan authored
Add support for VZ guest CP0_PWBase, CP0_PWField, CP0_PWSize, and CP0_PWCtl registers for controlling the guest hardware page table walker (HTW) present on P5600 and P6600 cores. These guest registers need initialising on R6, context switching, and exposing via the KVM ioctl API when they are present. Signed-off-by: -
James Hogan authored
Add support for VZ guest CP0_SegCtl0, CP0_SegCtl1, and CP0_SegCtl2 registers, as found on P5600 and P6600 cores. These guest registers need initialising, context switching, and exposing via the KVM ioctl API when they are present. They also require the GVA -> GPA translation code for handling a GVA root exception to be updated to interpret the segmentation registers and decode the faulting instruction enough to detect EVA memory access instructions. Signed-off-by: -
James Hogan authored
Add support for VZ guest CP0_ContextConfig and CP0_XContextConfig (MIPS64 only) registers, as found on P5600 and P6600 cores. These guest registers need initialising, context switching, and exposing via the KVM ioctl API when they are present. Signed-off-by: -
James Hogan authored
Add support for VZ guest CP0_BadInstr and CP0_BadInstrP registers, as found on most VZ capable cores. These guest registers need context switching, and exposing via the KVM ioctl API when they are present. Signed-off-by: -
James Hogan authored
Add support for the MIPS Virtualization (VZ) ASE to the MIPS KVM build system. For now KVM can only be configured for T&E or VZ and not both, but the design of the user facing APIs support the possibility of having both available, so this could change in future. Note that support for various optional guest features (some of which can't be turned off) are implemented in immediately following commits, so although it should now be possible to build VZ support, it may not work yet on your hardware. Signed-off-by: -
James Hogan authored
Add the main support for the MIPS Virtualization ASE (A.K.A. VZ) to MIPS KVM. The bulk of this work is in vz.c, with various new state and definitions elsewhere. Enough is implemented to be able to run on a minimal VZ core. Further patches will fill out support for guest features which are optional or can be disabled. Signed-off-by:
-
James Hogan authored
The general guest exit handler needs a few tweaks for VZ compared to trap & emulate, which for now are made directly depending on CONFIG_KVM_MIPS_VZ: - There is no need to re-enable the hardware page table walker (HTW), as it can be left enabled during guest mode operation with VZ. - There is no need to perform a privilege check, as any guest privilege violations should have already been detected by the hardware and triggered the appropriate guest exception. Signed-off-by: -
James Hogan authored
Ifdef out the trap & emulate CACHE instruction emulation functions for VZ. We will provide separate CACHE instruction emulation in vz.c, and we need to avoid linker errors due to the use of T&E specific MMU helpers. Signed-off-by: -
James Hogan authored
Update emulation of guest writes to CP0_Compare for VZ. There are two main differences compared to trap & emulate: - Writing to CP0_Compare in the VZ hardware guest context acks any pending timer, clearing CP0_Cause.TI. If we don't want an ack to take place we must carefully restore the TI bit if it was previously set. - Even with guest timer access disabled in CP0_GuestCtl0.GT, if the guest CP0_Count reaches the guest CP0_Compare the timer interrupt will assert. To prevent this we must set CP0_GTOffset to move the guest CP0_Count out of the way of the new guest CP0_Compare, either before or after depending on whether it is a forwards or backwards change. Signed-off-by: -
James Hogan authored
Add functions for MIPS VZ TLB management to tlb.c. kvm_vz_host_tlb_inv() will be used for invalidating root TLB entries after GPA page tables have been modified due to a KVM page fault. It arranges for a root GPA mapping to be flushed from the TLB, using the gpa_mm ASID or the current GuestID to do the probe. kvm_vz_local_flush_roottlb_all_guests() and kvm_vz_local_flush_guesttlb_all() flush all TLB entries in the corresponding TLB for guest mappings (GPA->RPA for root TLB with GuestID, and all entries for guest TLB). They will be used when starting a new GuestID cycle, when VZ hardware is enabled/disabled, and also when switching to a guest when the guest TLB contents may be stale or belong to a different VM. kvm_vz_guest_tlb_lookup() converts a guest virtual address to a guest physical address using the guest TLB. This will be used to decode guest virtual addresses which are sometimes provided by VZ hardware in CP0_BadVAddr for certain exceptions when the guest physical address is unavailable. kvm_vz_save_guesttlb() and kvm_vz_load_guesttlb() will be used to preserve wired guest VTLB entries while a guest isn't running. Signed-off-by:
-
James Hogan authored
Update MIPS KVM entry code to support VZ: - We need to set GuestCtl0.GM while in guest mode. - For cores supporting GuestID, we need to set the root GuestID to match the main GuestID while in guest mode so that the root TLB refill handler writes the correct GuestID into the TLB. - For cores without GuestID where the root ASID dealiases RVA/GPA mappings, we need to load that ASID from the gpa_mm rather than the per-VCPU guest_kernel_mm or guest_user_mm, since the root TLB maps guest physical addresses. We also need to restore the normal process ASID on exit. - The normal linux process pgd needs restoring on exit, as we can't leave the GPA mappings active for kernel code. - GuestCtl0 needs saving on exit for the GExcCode field, as it may be clobbered if a preemption occurs. We also need to move the TLB refill handler to the XTLB vector at offset 0x80 on 64-bit VZ kernels, as hardware will use Root.Status.KX to determine whether a TLB refill or XTLB Refill exception is to be taken on a root TLB miss from guest mode, and KX needs to be set for kernel code to be able to access the 64-bit segments. Signed-off-by: -
James Hogan authored
Abstract the MIPS KVM guest CP0 register access macros into inline functions which are generated by macros. This allows them to be generated differently for VZ, where they will usually need to access the hardware guest CP0 context rather than the saved values in RAM. Accessors for each individual register are generated using these macros: - __BUILD_KVM_*_SW() for registers which are not present in the VZ hardware guest context, so kvm_{read,write}_c0_guest_##name() will access the saved value in RAM regardless of whether VZ is enabled. - __BUILD_KVM_*_HW() for registers which are present in the VZ hardware guest context, so kvm_{read,write}_c0_guest_##name() will access the hardware register when VZ is enabled. These build the underlying accessors using further macros: - __BUILD_KVM_*_SAVED() builds e.g. kvm_{read,write}_sw_gc0_##name() functions for accessing the saved versions of the registers in RAM. This is used for implementing the common kvm_{read,write}_c0_guest_##name() accessors with T&E where registers are always stored in RAM, but are also available with VZ HW registers to allow them to be accessed while saved. - __BUILD_KVM_*_VZ() builds e.g. kvm_{read,write}_vz_gc0_##name() functions for accessing the VZ hardware guest context registers directly. This is used for implementing the common kvm_{read,write}_c0_guest_##name() accessors with VZ. - __BUILD_KVM_*_WRAP() builds wrappers with different names, which allows the common kvm_{read,write}_c0_guest_##name() functions to be implemented using the VZ accessors while still having the SAVED accessors available too. - __BUILD_KVM_SAVE_VZ() builds functions for saving and restoring VZ hardware guest context register state to RAM, improving conciseness of VZ context saving and restoring. Similar macros exist for generating modifiers (set, clear, change), either with a normal unlocked read/modify/write, or using atomic LL/SC sequences. These changes change the types of 32-bit registers to u32 instead of unsigned long, which requires some changes to printk() functions in MIPS KVM. Signed-off-by: -
James Hogan authored
Add a callback for MIPS KVM implementations to handle the VZ guest exit exception. Currently the trap & emulate implementation contains a stub which reports an internal error, but the callback will be used properly by the VZ implementation. Signed-off-by: -
James Hogan authored
Add an implementation callback for the kvm_arch_hardware_enable() and kvm_arch_hardware_disable() architecture functions, with simple stubs for trap & emulate. This is in preparation for VZ which will make use of them. Signed-off-by: -
James Hogan authored
Add an implementation callback for checking presence of KVM extensions. This allows implementation specific extensions to be provided without ifdefs in mips.c. Signed-off-by: -
James Hogan authored
Currently the software emulated timer is initialised to a frequency of 100MHz by kvm_mips_init_count(), but this isn't suitable for VZ where the frequency of the guest timer matches that of the host. Add a count_hz argument so the caller can specify the default frequency, and move the call from kvm_arch_vcpu_create() to the implementation specific vcpu_setup() callback, so that VZ can specify a different frequency. Signed-off-by: -
James Hogan authored
Add a new KVM_CAP_MIPS_64BIT capability to indicate that 64-bit MIPS guests are available and supported. In this case it should still be possible to run 32-bit guest code. If not available it won't be possible to run 64-bit guest code and the instructions may not be available, or the kernel may not support full context switching of 64-bit registers. Signed-off-by: -
James Hogan authored
Add new KVM_CAP_MIPS_VZ and KVM_CAP_MIPS_TE capabilities, and in order to allow MIPS KVM to support VZ without confusing old users (which expect the trap & emulate implementation), define and start checking KVM_CREATE_VM type codes. The codes available are: - KVM_VM_MIPS_TE = 0 This is the current value expected from the user, and will create a VM using trap & emulate in user mode, confined to the user mode address space. This may in future become unavailable if the kernel is only configured to support VZ, in which case the EINVAL error will be returned and KVM_CAP_MIPS_TE won't be available even though KVM_CAP_MIPS_VZ is. - KVM_VM_MIPS_VZ = 1 This can be provided when the KVM_CAP_MIPS_VZ capability is available to create a VM using VZ, with a fully virtualized guest virtual address space. If VZ support is unavailable in the kernel, the EINVAL error will be returned (although old kernels without the KVM_CAP_MIPS_VZ capability may well succeed and create a trap & emulate VM). This is designed to allow the desired implementation (T&E vs VZ) to be potentially chosen at runtime rather than being fixed in the kernel configuration. Signed-off-by: -
James Hogan authored
Extend MIPS KVM stats counters and kvm_transition trace event codes to cover hypervisor exceptions, which have their own GExcCode field in CP0_GuestCtl0 with up to 32 hypervisor exception cause codes. Signed-off-by: -
James Hogan authored
Update the implementation of kvm_lose_fpu() for VZ, where there is no need to enable the FPU/MSA in the root context if the FPU/MSA state is loaded but disabled in the guest context. The trap & emulate implementation needs to disable FPU/MSA in the root context when the guest disables them in order to catch the COP1 unusable or MSA disabled exception when they're used and pass it on to the guest. For VZ however as long as the context is loaded and enabled in the root context, the guest can enable and disable it in the guest context without the hypervisor having to do much, and will take guest exceptions without hypervisor intervention if used without being enabled in the guest context. Signed-off-by: -
James Hogan authored
Implement additional MMIO emulation for MIPS64, including 64-bit loads/stores, and 32-bit unsigned loads. These are only exposed on 64-bit VZ hosts. Signed-off-by: -
James Hogan authored
Refactor MIPS KVM MMIO load/store emulation to reduce code duplication. Each duplicate differed slightly anyway, and it will simplify adding 64-bit MMIO support for VZ. kvm_mips_emulate_store() and kvm_mips_emulate_load() can now return EMULATE_DO_MMIO (as possibly originally intended). We therefore stop calling either of these from kvm_mips_emulate_inst(), which is now only used by kvm_trap_emul_handle_cop_unusable() which is picky about return values. Signed-off-by: -
James Hogan authored
Emulate the HYPCALL instruction added in the VZ ASE and used by the MIPS paravirtualised guest support that is already merged. The new hypcall.c handles arguments and the return value. No actual hypercalls are yet supported, but this still allows us to safely step over hypercalls and set an error code in the return value for forward compatibility. Non-zero HYPCALL codes are not handled. We also document the hypercall ABI which asm/kvm_para.h uses. Signed-off-by:
-
James Hogan authored
Add a distinct UNIQUE_GUEST_ENTRYHI() macro for invalidation of guest TLB entries by KVM, using addresses in KSeg1 rather than KSeg0. This avoids conflicts with guest invalidation routines when there is no EHINV bit to mark the whole entry as invalid, avoiding guest machine check exceptions on Cavium Octeon III. Signed-off-by:
-
James Hogan authored
Add some missing guest accessors and register field definitions for KVM for MIPS VZ to make use of. Guest CP0_LLAddr register accessors and definitions for the LLB field allow KVM to clear the guest LLB to cancel in-progress LL/SC atomics on restore, and to emulate accesses by the guest to the CP0_LLAddr register. Bitwise modifiers and definitions for the guest CP0_Wired and CP0_Config1 registers allow KVM to modify fields within the CP0_Wired and CP0_Config1 registers. Finally a definition for the CP0_Config5.SBRI bit allows KVM to initialise and allow modification of the guest version of the SBRI bit. Signed-off-by:
-
James Hogan authored
Probe for availablility of M{T,F}HC0 instructions used with e.g. XPA in the VZ guest context, and make it available via cpu_guest_has_mvh. This will be helpful in properly emulating the MAAR registers in KVM for MIPS VZ. Signed-off-by: -
James Hogan authored
Probe for presence of guest CP0_UserLocal register and expose via cpu_guest_has_userlocal. This register is optional pre-r6, so this will allow KVM to only save/restore/expose the guest CP0_UserLocal register if it exists. Signed-off-by:
-
