• Vlastimil Babka's avatar
    mm: wake kcompactd before kswapd's short sleep · fd901c95
    Vlastimil Babka authored
    When kswapd goes to sleep it checks if the node is balanced and at first
    it sleeps only for HZ/10 time, then rechecks if the node is still
    balanced and nobody has woken it during the initial sleep.  Only then it
    goes fully sleep until an allocation slowpath wakes it up again.
    
    For higher-order allocations, waking up kcompactd is done only before
    the full sleep.  This turns out to be an issue in case another
    high-order allocation fails during the initial sleep.  It will wake
    kswapd up, however kswapd considers the zone balanced from the order-0
    perspective, and will just quickly try to sleep again.  So if there's a
    longer stream of high-order allocations hitting the slowpath and waking
    up kswapd, it might never actually wake up kcompactd, which may be
    considered a regression from kswapd-based compaction.  In the worst
    case, it might be that a single allocation that cannot direct
    reclaim/compact itself is waking kswapd in the retry loop and preventing
    kcompactd from being woken up and unblocking it.
    
    This patch makes sure kcompactd is woken up in such situations by simply
    moving the wakeup before the short initial sleep.  More efficient
    solution would be to wake kcompactd immediately instead of kswapd if the
    node is already order-0 balanced, but in that case we should also move
    reset_isolation_suitable() call to kcompactd so it's not adding to the
    allocator's latency.  Since it's late in the 4.6 cycle, let's go with
    the simpler change for now.
    
    Fixes: accf6242 ("mm, kswapd: replace kswapd compaction with waking up kcompactd")
    Signed-off-by: default avatarVlastimil Babka <vbabka@suse.cz>
    Cc: Andrea Arcangeli <aarcange@redhat.com>
    Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
    Cc: Rik van Riel <riel@redhat.com>
    Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
    Cc: Mel Gorman <mgorman@techsingularity.net>
    Cc: David Rientjes <rientjes@google.com>
    Cc: Michal Hocko <mhocko@suse.com>
    Cc: Johannes Weiner <hannes@cmpxchg.org>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    fd901c95
vmscan.c 110 KB