• Konstantin Khlebnikov's avatar
    mm/balloon_compaction: redesign ballooned pages management · 514488e2
    Konstantin Khlebnikov authored
    commit d6d86c0a upstream.
    
    Sasha Levin reported KASAN splash inside isolate_migratepages_range().
    Problem is in the function __is_movable_balloon_page() which tests
    AS_BALLOON_MAP in page->mapping->flags.  This function has no protection
    against anonymous pages.  As result it tried to check address space flags
    inside struct anon_vma.
    
    Further investigation shows more problems in current implementation:
    
    * Special branch in __unmap_and_move() never works:
      balloon_page_movable() checks page flags and page_count.  In
      __unmap_and_move() page is locked, reference counter is elevated, thus
      balloon_page_movable() always fails.  As a result execution goes to the
      normal migration path.  virtballoon_migratepage() returns
      MIGRATEPAGE_BALLOON_SUCCESS instead of MIGRATEPAGE_SUCCESS,
      move_to_new_page() thinks this is an error code and assigns
      newpage->mapping to NULL.  Newly migrated page lose connectivity with
      balloon an all ability for further migration.
    
    * lru_lock erroneously required in isolate_migratepages_range() for
      isolation ballooned page.  This function releases lru_lock periodically,
      this makes migration mostly impossible for some pages.
    
    * balloon_page_dequeue have a tight race with balloon_page_isolate:
      balloon_page_isolate could be executed in parallel with dequeue between
      picking page from list and locking page_lock.  Race is rare because they
      use trylock_page() for locking.
    
    This patch fixes all of them.
    
    Instead of fake mapping with special flag this patch uses special state of
    page->_mapcount: PAGE_BALLOON_MAPCOUNT_VALUE = -256.  Buddy allocator uses
    PAGE_BUDDY_MAPCOUNT_VALUE = -128 for similar purpose.  Storing mark
    directly in struct page makes everything safer and easier.
    
    PagePrivate is used to mark pages present in page list (i.e.  not
    isolated, like PageLRU for normal pages).  It replaces special rules for
    reference counter and makes balloon migration similar to migration of
    normal pages.  This flag is protected by page_lock together with link to
    the balloon device.
    Signed-off-by: default avatarKonstantin Khlebnikov <k.khlebnikov@samsung.com>
    Reported-by: default avatarSasha Levin <sasha.levin@oracle.com>
    Link: http://lkml.kernel.org/p/53E6CEAA.9020105@oracle.com
    Cc: Rafael Aquini <aquini@redhat.com>
    Cc: Andrey Ryabinin <ryabinin.a.a@gmail.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    [bwh: Backported to 3.16:
     - Remove an additional check for MIGRATEPAGE_BALLOON_SUCCESS in
       __unmap_and_move()
     - Adjust context]
    Signed-off-by: default avatarBen Hutchings <ben@decadent.org.uk>
    Cc: Jiri Kosina <jkosina@suse.cz>
    Cc: jian wang <wangjian@bytedance.com>
    514488e2
migrate.c 49.9 KB