• Andrea Arcangeli's avatar
    thp: avoid atomic64_read in pmd_read_atomic for 32bit PAE · e4eed03f
    Andrea Arcangeli authored
    In the x86 32bit PAE CONFIG_TRANSPARENT_HUGEPAGE=y case while holding the
    mmap_sem for reading, cmpxchg8b cannot be used to read pmd contents under
    Xen.
    
    So instead of dealing only with "consistent" pmdvals in
    pmd_none_or_trans_huge_or_clear_bad() (which would be conceptually
    simpler) we let pmd_none_or_trans_huge_or_clear_bad() deal with pmdvals
    where the low 32bit and high 32bit could be inconsistent (to avoid having
    to use cmpxchg8b).
    
    The only guarantee we get from pmd_read_atomic is that if the low part of
    the pmd was found null, the high part will be null too (so the pmd will be
    considered unstable).  And if the low part of the pmd is found "stable"
    later, then it means the whole pmd was read atomically (because after a
    pmd is stable, neither MADV_DONTNEED nor page faults can alter it anymore,
    and we read the high part after the low part).
    
    In the 32bit PAE x86 case, it is enough to read the low part of the pmdval
    atomically to declare the pmd as "stable" and that's true for THP and no
    THP, furthermore in the THP case we also have a barrier() that will
    prevent any inconsistent pmdvals to be cached by a later re-read of the
    *pmd.
    Signed-off-by: default avatarAndrea Arcangeli <aarcange@redhat.com>
    Cc: Jonathan Nieder <jrnieder@gmail.com>
    Cc: Ulrich Obergfell <uobergfe@redhat.com>
    Cc: Mel Gorman <mgorman@suse.de>
    Cc: Hugh Dickins <hughd@google.com>
    Cc: Larry Woodman <lwoodman@redhat.com>
    Cc: Petr Matousek <pmatouse@redhat.com>
    Cc: Rik van Riel <riel@redhat.com>
    Cc: Jan Beulich <jbeulich@suse.com>
    Cc: KOSAKI Motohiro <kosaki.motohiro@gmail.com>
    Tested-by: default avatarAndrew Jones <drjones@redhat.com>
    Cc: <stable@vger.kernel.org>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    e4eed03f
pgtable-3level.h 6.2 KB