• Mel Gorman's avatar
    mm: numa: serialise parallel get_user_page against THP migration · 2b4847e7
    Mel Gorman authored
    Base pages are unmapped and flushed from cache and TLB during normal
    page migration and replaced with a migration entry that causes any
    parallel NUMA hinting fault or gup to block until migration completes.
    
    THP does not unmap pages due to a lack of support for migration entries
    at a PMD level.  This allows races with get_user_pages and
    get_user_pages_fast which commit 3f926ab9 ("mm: Close races between
    THP migration and PMD numa clearing") made worse by introducing a
    pmd_clear_flush().
    
    This patch forces get_user_page (fast and normal) on a pmd_numa page to
    go through the slow get_user_page path where it will serialise against
    THP migration and properly account for the NUMA hinting fault.  On the
    migration side the page table lock is taken for each PTE update.
    Signed-off-by: default avatarMel Gorman <mgorman@suse.de>
    Reviewed-by: default avatarRik van Riel <riel@redhat.com>
    Cc: Alex Thorlton <athorlton@sgi.com>
    Cc: <stable@vger.kernel.org>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    2b4847e7
gup.c 10.5 KB