• Jérôme Glisse's avatar
    mm/mmu_notifier: avoid call to invalidate_range() in range_end() · 4645b9fe
    Jérôme Glisse authored
    This is an optimization patch that only affect mmu_notifier users which
    rely on the invalidate_range() callback.  This patch avoids calling that
    callback twice in a row from inside __mmu_notifier_invalidate_range_end
    
    Existing pattern (before this patch):
        mmu_notifier_invalidate_range_start()
            pte/pmd/pud_clear_flush_notify()
                mmu_notifier_invalidate_range()
        mmu_notifier_invalidate_range_end()
            mmu_notifier_invalidate_range()
    
    New pattern (after this patch):
        mmu_notifier_invalidate_range_start()
            pte/pmd/pud_clear_flush_notify()
                mmu_notifier_invalidate_range()
        mmu_notifier_invalidate_range_only_end()
    
    We call the invalidate_range callback after clearing the page table
    under the page table lock and we skip the call to invalidate_range
    inside the __mmu_notifier_invalidate_range_end() function.
    
    Idea from Andrea Arcangeli
    
    Link: http://lkml.kernel.org/r/20171017031003.7481-3-jglisse@redhat.comSigned-off-by: default avatarJérôme Glisse <jglisse@redhat.com>
    Cc: Andrea Arcangeli <aarcange@redhat.com>
    Cc: Joerg Roedel <jroedel@suse.de>
    Cc: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
    Cc: David Woodhouse <dwmw2@infradead.org>
    Cc: Alistair Popple <alistair@popple.id.au>
    Cc: Michael Ellerman <mpe@ellerman.id.au>
    Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
    Cc: Stephen Rothwell <sfr@canb.auug.org.au>
    Cc: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
    Cc: Nadav Amit <nadav.amit@gmail.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    4645b9fe
migrate.c 77.1 KB