• Oliver Upton's avatar
    KVM: arm64: Don't split hugepages outside of MMU write lock · f587661f
    Oliver Upton authored
    It is possible to take a stage-2 permission fault on a page larger than
    PAGE_SIZE. For example, when running a guest backed by 2M HugeTLB, KVM
    eagerly maps at the largest possible block size. When dirty logging is
    enabled on a memslot, KVM does *not* eagerly split these 2M stage-2
    mappings and instead clears the write bit on the pte.
    
    Since dirty logging is always performed at PAGE_SIZE granularity, KVM
    lazily splits these 2M block mappings down to PAGE_SIZE in the stage-2
    fault handler. This operation must be done under the write lock. Since
    commit f783ef1c ("KVM: arm64: Add fast path to handle permission
    relaxation during dirty logging"), the stage-2 fault handler
    conditionally takes the read lock on permission faults with dirty
    logging enabled. To that end, it is possible to split a 2M block mapping
    while only holding the read lock.
    
    The problem is demonstrated by running kvm_page_table_test with 2M
    anonymous HugeTLB, which splats like so:
    
      WARNING: CPU: 5 PID: 15276 at arch/arm64/kvm/hyp/pgtable.c:153 stage2_map_walk_leaf+0x124/0x158
    
      [...]
    
      Call trace:
      stage2_map_walk_leaf+0x124/0x158
      stage2_map_walker+0x5c/0xf0
      __kvm_pgtable_walk+0x100/0x1d4
      __kvm_pgtable_walk+0x140/0x1d4
      __kvm_pgtable_walk+0x140/0x1d4
      kvm_pgtable_walk+0xa0/0xf8
      kvm_pgtable_stage2_map+0x15c/0x198
      user_mem_abort+0x56c/0x838
      kvm_handle_guest_abort+0x1fc/0x2a4
      handle_exit+0xa4/0x120
      kvm_arch_vcpu_ioctl_run+0x200/0x448
      kvm_vcpu_ioctl+0x588/0x664
      __arm64_sys_ioctl+0x9c/0xd4
      invoke_syscall+0x4c/0x144
      el0_svc_common+0xc4/0x190
      do_el0_svc+0x30/0x8c
      el0_svc+0x28/0xcc
      el0t_64_sync_handler+0x84/0xe4
      el0t_64_sync+0x1a4/0x1a8
    
    Fix the issue by only acquiring the read lock if the guest faulted on a
    PAGE_SIZE granule w/ dirty logging enabled. Add a WARN to catch locking
    bugs in future changes.
    
    Fixes: f783ef1c ("KVM: arm64: Add fast path to handle permission relaxation during dirty logging")
    Cc: Jing Zhang <jingzhangos@google.com>
    Signed-off-by: default avatarOliver Upton <oupton@google.com>
    Reviewed-by: default avatarReiji Watanabe <reijiw@google.com>
    Signed-off-by: default avatarMarc Zyngier <maz@kernel.org>
    Link: https://lore.kernel.org/r/20220401194652.950240-1-oupton@google.com
    f587661f
mmu.c 47.3 KB