• Will Deacon's avatar
    mm: page_vma_mapped: ensure pmd is loaded with READ_ONCE outside of lock · a7b10095
    Will Deacon authored
    Loading the pmd without holding the pmd_lock exposes us to races with
    concurrent updaters of the page tables but, worse still, it also allows
    the compiler to cache the pmd value in a register and reuse it later on,
    even if we've performed a READ_ONCE in between and seen a more recent
    value.
    
    In the case of page_vma_mapped_walk, this leads to the following crash
    when the pmd loaded for the initial pmd_trans_huge check is all zeroes
    and a subsequent valid table entry is loaded by check_pmd.  We then
    proceed into map_pte, but the compiler re-uses the zero entry inside
    pte_offset_map, resulting in a junk pointer being installed in
    pvmw->pte:
    
      PC is at check_pte+0x20/0x170
      LR is at page_vma_mapped_walk+0x2e0/0x540
      [...]
      Process doio (pid: 2463, stack limit = 0xffff00000f2e8000)
      Call trace:
        check_pte+0x20/0x170
        page_vma_mapped_walk+0x2e0/0x540
        page_mkclean_one+0xac/0x278
        rmap_walk_file+0xf0/0x238
        rmap_walk+0x64/0xa0
        page_mkclean+0x90/0xa8
        clear_page_dirty_for_io+0x84/0x2a8
        mpage_submit_page+0x34/0x98
        mpage_process_page_bufs+0x164/0x170
        mpage_prepare_extent_to_map+0x134/0x2b8
        ext4_writepages+0x484/0xe30
        do_writepages+0x44/0xe8
        __filemap_fdatawrite_range+0xbc/0x110
        file_write_and_wait_range+0x48/0xd8
        ext4_sync_file+0x80/0x4b8
        vfs_fsync_range+0x64/0xc0
        SyS_msync+0x194/0x1e8
    
    This patch fixes the problem by ensuring that READ_ONCE is used before
    the initial checks on the pmd, and this value is subsequently used when
    checking whether or not the pmd is present.  pmd_check is removed and
    the pmd_present check is inlined directly.
    
    Link: http://lkml.kernel.org/r/1507222630-5839-1-git-send-email-will.deacon@arm.com
    Fixes: f27176cf ("mm: convert page_mkclean_one() to use page_vma_mapped_walk()")
    Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
    Tested-by: default avatarYury Norov <ynorov@caviumnetworks.com>
    Tested-by: default avatarRichard Ruigrok <rruigrok@codeaurora.org>
    Acked-by: default avatarKirill A. Shutemov <kirill.shutemov@linux.intel.com>
    Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: <stable@vger.kernel.org>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    a7b10095
page_vma_mapped.c 6.3 KB