• Huang Ying's avatar
    mm, gup: prevent pmd checking race in follow_pmd_mask() · 68827280
    Huang Ying authored
    mmap_sem will be read locked when calling follow_pmd_mask().  But this
    cannot prevent PMD from being changed for all cases when PTL is
    unlocked, for example, from pmd_trans_huge() to pmd_none() via
    MADV_DONTNEED.  So it is possible for the pmd_present() check in
    follow_pmd_mask() to encounter an invalid PMD.  This may cause an
    incorrect VM_BUG_ON() or an infinite loop.  Fix this by reading the PMD
    entry into a local variable with READ_ONCE() and checking the local
    variable and pmd_none() in the retry loop.
    
    As Kirill pointed out, with PTL unlocked, the *pmd may be changed under
    us, so reading it directly again and again may incur weird bugs.  So
    although using *pmd directly other than for pmd_present() checking may
    be safe, it is still better to replace them to read *pmd once and check
    the local variable multiple times.
    
    When PTL unlocked, replace all *pmd with local variable was suggested by
    Kirill.
    
    Link: http://lkml.kernel.org/r/20180419083514.1365-1-ying.huang@intel.comSigned-off-by: default avatar"Huang, Ying" <ying.huang@intel.com>
    Reviewed-by: default avatarZi Yan <zi.yan@cs.rutgers.edu>
    Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
    Cc: Al Viro <viro@zeniv.linux.org.uk>
    Cc: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
    Cc: Dan Williams <dan.j.williams@intel.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    68827280
gup.c 50.3 KB