• Mark Rutland's avatar
    arm64: Avoid cpus_have_const_cap() for ARM64_HAS_PAN · 53d62e99
    Mark Rutland authored
    In system_uses_hw_pan() we use cpus_have_const_cap() to check for
    ARM64_HAS_PAN, but this is only necessary so that the
    system_uses_ttbr0_pan() check in setup_cpu_features() can run prior to
    alternatives being patched, and otherwise this is not necessary and
    alternative_has_cap_*() would be preferable.
    
    For historical reasons, cpus_have_const_cap() is more complicated than
    it needs to be. Before cpucaps are finalized, it will perform a bitmap
    test of the system_cpucaps bitmap, and once cpucaps are finalized it
    will use an alternative branch. This used to be necessary to handle some
    race conditions in the window between cpucap detection and the
    subsequent patching of alternatives and static branches, where different
    branches could be out-of-sync with one another (or w.r.t. alternative
    sequences). Now that we use alternative branches instead of static
    branches, these are all patched atomically w.r.t. one another, and there
    are only a handful of cases that need special care in the window between
    cpucap detection and alternative patching.
    
    Due to the above, it would be nice to remove cpus_have_const_cap(), and
    migrate callers over to alternative_has_cap_*(), cpus_have_final_cap(),
    or cpus_have_cap() depending on when their requirements. This will
    remove redundant instructions and improve code generation, and will make
    it easier to determine how each callsite will behave before, during, and
    after alternative patching.
    
    The ARM64_HAS_PAN cpucap is used by system_uses_hw_pan() and
    system_uses_ttbr0_pan() depending on whether CONFIG_ARM64_SW_TTBR0_PAN
    is selected, and:
    
    * We only use system_uses_hw_pan() directly in __sdei_handler(), which
      isn't reachable until after alternatives have been patched, and for
      this it is safe to use alternative_has_cap_*().
    
    * We use system_uses_ttbr0_pan() in a few places:
    
      - In check_and_switch_context() and cpu_uninstall_idmap(), which will
        defer installing a translation table into TTBR0 when the
        ARM64_HAS_PAN cpucap is not detected.
    
        Prior to patching alternatives, all CPUs will be using init_mm with
        the reserved ttbr0 translation tables install in TTBR0, so these can
        safely use alternative_has_cap_*().
    
      - In update_saved_ttbr0(), which will only save the active TTBR0 into
        a per-thread variable when the ARM64_HAS_PAN cpucap is not detected.
    
        Prior to patching alternatives, all CPUs will be using init_mm with
        the reserved ttbr0 translation tables install in TTBR0, so these can
        safely use alternative_has_cap_*().
    
      - In efi_set_pgd(), which will handle check_and_switch_context()
        deferring the installation of TTBR0 when TTBR0 PAN is detected.
    
        The EFI runtime services are not initialized until after
        alternatives have been patched, and so this can safely use
        alternative_has_cap_*() or cpus_have_final_cap().
    
      - In uaccess_ttbr0_disable() and uaccess_ttbr0_enable(), where we'll
        avoid installing/uninstalling a translation table in TTBR0 when
        ARM64_HAS_PAN is detected.
    
        Prior to patching alternatives we will not perform any uaccess and
        will not call uaccess_ttbr0_disable() or uaccess_ttbr0_enable(), and
        so these can safely use alternative_has_cap_*() or
        cpus_have_final_cap().
    
      - In is_el1_permission_fault() where we will consider a translation
        fault on a TTBR0 address to be a permission fault when ARM64_HAS_PAN
        is not detected *and* we have set the PAN bit in the SPSR (which
        tells us that in the interrupted context, TTBR0 pointed at the
        reserved zero ttbr).
    
        In the window between detecting system cpucaps and patching
        alternatives we should not perform any accesses to TTBR0 addresses,
        and no userspace translation tables exist until after patching
        alternatives. Thus it is safe for this to use alternative_has_cap*().
    
    This patch replaces the use of cpus_have_const_cap() with
    alternative_has_cap_unlikely(), which will avoid generating code to test
    the system_cpucaps bitmap and should be better for all subsequent calls
    at runtime.
    
    So that the check for TTBR0 PAN in setup_cpu_features() can run prior to
    alternatives being patched, the call to system_uses_ttbr0_pan() is
    replaced with an explicit check of the ARM64_HAS_PAN bit in the
    system_cpucaps bitmap.
    Signed-off-by: default avatarMark Rutland <mark.rutland@arm.com>
    Cc: James Morse <james.morse@arm.com>
    Cc: Marc Zyngier <maz@kernel.org>
    Cc: Suzuki K Poulose <suzuki.poulose@arm.com>
    Cc: Will Deacon <will@kernel.org>
    Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
    53d62e99
cpufeature.h 29.7 KB