• David Rientjes's avatar
    mm: close PageTail race · 668f9abb
    David Rientjes authored
    Commit bf6bddf1 ("mm: introduce compaction and migration for
    ballooned pages") introduces page_count(page) into memory compaction
    which dereferences page->first_page if PageTail(page).
    
    This results in a very rare NULL pointer dereference on the
    aforementioned page_count(page).  Indeed, anything that does
    compound_head(), including page_count() is susceptible to racing with
    prep_compound_page() and seeing a NULL or dangling page->first_page
    pointer.
    
    This patch uses Andrea's implementation of compound_trans_head() that
    deals with such a race and makes it the default compound_head()
    implementation.  This includes a read memory barrier that ensures that
    if PageTail(head) is true that we return a head page that is neither
    NULL nor dangling.  The patch then adds a store memory barrier to
    prep_compound_page() to ensure page->first_page is set.
    
    This is the safest way to ensure we see the head page that we are
    expecting, PageTail(page) is already in the unlikely() path and the
    memory barriers are unfortunately required.
    
    Hugetlbfs is the exception, we don't enforce a store memory barrier
    during init since no race is possible.
    Signed-off-by: default avatarDavid Rientjes <rientjes@google.com>
    Cc: Holger Kiehl <Holger.Kiehl@dwd.de>
    Cc: Christoph Lameter <cl@linux.com>
    Cc: Rafael Aquini <aquini@redhat.com>
    Cc: Vlastimil Babka <vbabka@suse.cz>
    Cc: Michal Hocko <mhocko@suse.cz>
    Cc: Mel Gorman <mgorman@suse.de>
    Cc: Andrea Arcangeli <aarcange@redhat.com>
    Cc: Rik van Riel <riel@redhat.com>
    Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
    Cc: <stable@vger.kernel.org>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    668f9abb
ksm.c 63.4 KB