• Nick Piggin's avatar
    mlock: mlocked pages are unevictable · b291f000
    Nick Piggin authored
    Make sure that mlocked pages also live on the unevictable LRU, so kswapd
    will not scan them over and over again.
    
    This is achieved through various strategies:
    
    1) add yet another page flag--PG_mlocked--to indicate that
       the page is locked for efficient testing in vmscan and,
       optionally, fault path.  This allows early culling of
       unevictable pages, preventing them from getting to
       page_referenced()/try_to_unmap().  Also allows separate
       accounting of mlock'd pages, as Nick's original patch
       did.
    
       Note:  Nick's original mlock patch used a PG_mlocked
       flag.  I had removed this in favor of the PG_unevictable
       flag + an mlock_count [new page struct member].  I
       restored the PG_mlocked flag to eliminate the new
       count field.
    
    2) add the mlock/unevictable infrastructure to mm/mlock.c,
       with internal APIs in mm/internal.h.  This is a rework
       of Nick's original patch to these files, taking into
       account that mlocked pages are now kept on unevictable
       LRU list.
    
    3) update vmscan.c:page_evictable() to check PageMlocked()
       and, if vma passed in, the vm_flags.  Note that the vma
       will only be passed in for new pages in the fault path;
       and then only if the "cull unevictable pages in fault
       path" patch is included.
    
    4) add try_to_unlock() to rmap.c to walk a page's rmap and
       ClearPageMlocked() if no other vmas have it mlocked.
       Reuses as much of try_to_unmap() as possible.  This
       effectively replaces the use of one of the lru list links
       as an mlock count.  If this mechanism let's pages in mlocked
       vmas leak through w/o PG_mlocked set [I don't know that it
       does], we should catch them later in try_to_unmap().  One
       hopes this will be rare, as it will be relatively expensive.
    
    Original mm/internal.h, mm/rmap.c and mm/mlock.c changes:
    Signed-off-by: default avatarNick Piggin <npiggin@suse.de>
    
    splitlru: introduce __get_user_pages():
    
      New munlock processing need to GUP_FLAGS_IGNORE_VMA_PERMISSIONS.
      because current get_user_pages() can't grab PROT_NONE pages theresore it
      cause PROT_NONE pages can't munlock.
    
    [akpm@linux-foundation.org: fix this for pagemap-pass-mm-into-pagewalkers.patch]
    [akpm@linux-foundation.org: untangle patch interdependencies]
    [akpm@linux-foundation.org: fix things after out-of-order merging]
    [hugh@veritas.com: fix page-flags mess]
    [lee.schermerhorn@hp.com: fix munlock page table walk - now requires 'mm']
    [kosaki.motohiro@jp.fujitsu.com: build fix]
    [kosaki.motohiro@jp.fujitsu.com: fix truncate race and sevaral comments]
    [kosaki.motohiro@jp.fujitsu.com: splitlru: introduce __get_user_pages()]
    Signed-off-by: default avatarKOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
    Signed-off-by: default avatarRik van Riel <riel@redhat.com>
    Signed-off-by: default avatarLee Schermerhorn <lee.schermerhorn@hp.com>
    Cc: Nick Piggin <npiggin@suse.de>
    Cc: Dave Hansen <dave@linux.vnet.ibm.com>
    Cc: Matt Mackall <mpm@selenic.com>
    Signed-off-by: default avatarHugh Dickins <hugh@veritas.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    b291f000
rmap.c 34.4 KB