• Sean Christopherson's avatar
    KVM: x86/mmu: Move calls to thp_adjust() down a level · 4cd071d1
    Sean Christopherson authored
    Move the calls to thp_adjust() down a level from the page fault handlers
    to the map/fetch helpers and remove the page count shuffling done in
    thp_adjust().
    
    Despite holding a reference to the underlying page while processing a
    page fault, the page fault flows don't actually rely on holding a
    reference to the page when thp_adjust() is called.  At that point, the
    fault handlers hold mmu_lock, which prevents mmu_notifier from completing
    any invalidations, and have verified no invalidations from mmu_notifier
    have occurred since the page reference was acquired (which is done prior
    to taking mmu_lock).
    
    The kvm_release_pfn_clean()/kvm_get_pfn() dance in thp_adjust() is a
    quirk that is necessitated because thp_adjust() modifies the pfn that is
    consumed by its caller.  Because the page fault handlers call
    kvm_release_pfn_clean() on said pfn, thp_adjust() needs to transfer the
    reference to the correct pfn purely for correctness when the pfn is
    released.
    
    Calling thp_adjust() from __direct_map() and FNAME(fetch) means the pfn
    adjustment doesn't change the pfn as seen by the page fault handlers,
    i.e. the pfn released by the page fault handlers is the same pfn that
    was returned by gfn_to_pfn().
    Signed-off-by: default avatarSean Christopherson <sean.j.christopherson@intel.com>
    Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
    4cd071d1
mmu.c 170 KB