• Michel Lespinasse's avatar
    mm: avoid taking rmap locks in move_ptes() · 38a76013
    Michel Lespinasse authored
    During mremap(), the destination VMA is generally placed after the
    original vma in rmap traversal order: in move_vma(), we always have
    new_pgoff >= vma->vm_pgoff, and as a result new_vma->vm_pgoff >=
    vma->vm_pgoff unless vma_merge() merged the new vma with an adjacent one.
    
    When the destination VMA is placed after the original in rmap traversal
    order, we can avoid taking the rmap locks in move_ptes().
    
    Essentially, this reintroduces the optimization that had been disabled in
    "mm anon rmap: remove anon_vma_moveto_tail".  The difference is that we
    don't try to impose the rmap traversal order; instead we just rely on
    things being in the desired order in the common case and fall back to
    taking locks in the uncommon case.  Also we skip the i_mmap_mutex in
    addition to the anon_vma lock: in both cases, the vmas are traversed in
    increasing vm_pgoff order with ties resolved in tree insertion order.
    Signed-off-by: default avatarMichel Lespinasse <walken@google.com>
    Cc: Andrea Arcangeli <aarcange@redhat.com>
    Cc: Rik van Riel <riel@redhat.com>
    Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
    Cc: Daniel Santos <daniel.santos@pobox.com>
    Cc: Hugh Dickins <hughd@google.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    38a76013
mmap.c 71.3 KB