• Vlastimil Babka's avatar
    mm, compaction: pass classzone_idx and alloc_flags to watermark checking · ebff3980
    Vlastimil Babka authored
    Compaction relies on zone watermark checks for decisions such as if it's
    worth to start compacting in compaction_suitable() or whether compaction
    should stop in compact_finished().  The watermark checks take
    classzone_idx and alloc_flags parameters, which are related to the memory
    allocation request.  But from the context of compaction they are currently
    passed as 0, including the direct compaction which is invoked to satisfy
    the allocation request, and could therefore know the proper values.
    
    The lack of proper values can lead to mismatch between decisions taken
    during compaction and decisions related to the allocation request.  Lack
    of proper classzone_idx value means that lowmem_reserve is not taken into
    account.  This has manifested (during recent changes to deferred
    compaction) when DMA zone was used as fallback for preferred Normal zone.
    compaction_suitable() without proper classzone_idx would think that the
    watermarks are already satisfied, but watermark check in
    get_page_from_freelist() would fail.  Because of this problem, deferring
    compaction has extra complexity that can be removed in the following
    patch.
    
    The issue (not confirmed in practice) with missing alloc_flags is opposite
    in nature.  For allocations that include ALLOC_HIGH, ALLOC_HIGHER or
    ALLOC_CMA in alloc_flags (the last includes all MOVABLE allocations on
    CMA-enabled systems) the watermark checking in compaction with 0 passed
    will be stricter than in get_page_from_freelist().  In these cases
    compaction might be running for a longer time than is really needed.
    
    Another issue compaction_suitable() is that the check for "does the zone
    need compaction at all?" comes only after the check "does the zone have
    enough free free pages to succeed compaction".  The latter considers extra
    pages for migration and can therefore in some situations fail and return
    COMPACT_SKIPPED, although the high-order allocation would succeed and we
    should return COMPACT_PARTIAL.
    
    This patch fixes these problems by adding alloc_flags and classzone_idx to
    struct compact_control and related functions involved in direct compaction
    and watermark checking.  Where possible, all other callers of
    compaction_suitable() pass proper values where those are known.  This is
    currently limited to classzone_idx, which is sometimes known in kswapd
    context.  However, the direct reclaim callers should_continue_reclaim()
    and compaction_ready() do not currently know the proper values, so the
    coordination between reclaim and compaction may still not be as accurate
    as it could.  This can be fixed later, if it's shown to be an issue.
    
    Additionaly the checks in compact_suitable() are reordered to address the
    second issue described above.
    
    The effect of this patch should be slightly better high-order allocation
    success rates and/or less compaction overhead, depending on the type of
    allocations and presence of CMA.  It allows simplifying deferred
    compaction code in a followup patch.
    
    When testing with stress-highalloc, there was some slight improvement
    (which might be just due to variance) in success rates of non-THP-like
    allocations.
    Signed-off-by: default avatarVlastimil Babka <vbabka@suse.cz>
    Cc: Minchan Kim <minchan@kernel.org>
    Cc: Mel Gorman <mgorman@suse.de>
    Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
    Cc: Michal Nazarewicz <mina86@mina86.com>
    Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
    Cc: Christoph Lameter <cl@linux.com>
    Acked-by: default avatarRik van Riel <riel@redhat.com>
    Cc: David Rientjes <rientjes@google.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    ebff3980
compaction.c 42.3 KB