• Rik van Riel's avatar
    mm: fix TLB flush race between migration, and change_protection_range · 20841405
    Rik van Riel authored
    There are a few subtle races, between change_protection_range (used by
    mprotect and change_prot_numa) on one side, and NUMA page migration and
    compaction on the other side.
    
    The basic race is that there is a time window between when the PTE gets
    made non-present (PROT_NONE or NUMA), and the TLB is flushed.
    
    During that time, a CPU may continue writing to the page.
    
    This is fine most of the time, however compaction or the NUMA migration
    code may come in, and migrate the page away.
    
    When that happens, the CPU may continue writing, through the cached
    translation, to what is no longer the current memory location of the
    process.
    
    This only affects x86, which has a somewhat optimistic pte_accessible.
    All other architectures appear to be safe, and will either always flush,
    or flush whenever there is a valid mapping, even with no permissions
    (SPARC).
    
    The basic race looks like this:
    
    CPU A			CPU B			CPU C
    
    						load TLB entry
    make entry PTE/PMD_NUMA
    			fault on entry
    						read/write old page
    			start migrating page
    			change PTE/PMD to new page
    						read/write old page [*]
    flush TLB
    						reload TLB from new entry
    						read/write new page
    						lose data
    
    [*] the old page may belong to a new user at this point!
    
    The obvious fix is to flush remote TLB entries, by making sure that
    pte_accessible aware of the fact that PROT_NONE and PROT_NUMA memory may
    still be accessible if there is a TLB flush pending for the mm.
    
    This should fix both NUMA migration and compaction.
    
    [mgorman@suse.de: fix build]
    Signed-off-by: default avatarRik van Riel <riel@redhat.com>
    Signed-off-by: default avatarMel Gorman <mgorman@suse.de>
    Cc: Alex Thorlton <athorlton@sgi.com>
    Cc: <stable@vger.kernel.org>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    20841405
fork.c 45.7 KB