• Alex Van Brunt's avatar
    arm64: mm: Don't wait for completion of TLB invalidation when page aging · 3403e56b
    Alex Van Brunt authored
    When transitioning a PTE from young to old as part of page aging, we
    can avoid waiting for the TLB invalidation to complete and therefore
    drop the subsequent DSB instruction. Whilst this opens up a race with
    page reclaim, where a PTE in active use via a stale, young TLB entry
    does not update the underlying descriptor, the worst thing that happens
    is that the page is reclaimed and then immediately faulted back in.
    
    Given that we have a DSB in our context-switch path, the window for a
    spurious reclaim is fairly limited and eliding the barrier claims to
    boost NVMe/SSD accesses by over 10% on some platforms.
    
    A similar optimisation was made for x86 in commit b13b1d2d ("x86/mm:
    In the PTE swapout page reclaim case clear the accessed bit instead of
    flushing the TLB").
    Signed-off-by: default avatarAlex Van Brunt <avanbrunt@nvidia.com>
    Signed-off-by: default avatarAshish Mhetre <amhetre@nvidia.com>
    [will: rewrote patch]
    Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
    3403e56b
pgtable.h 24.5 KB