• Vlastimil Babka's avatar
    mm, compaction: more robust check for scanners meeting · f2849aa0
    Vlastimil Babka authored
    Assorted compaction cleanups and optimizations.  The interesting patches
    are 4 and 5.  In 4, skipping of compound pages in single iteration is
    improved for migration scanner, so it works also for !PageLRU compound
    pages such as hugetlbfs, slab etc.  Patch 5 introduces this kind of
    skipping in the free scanner.  The trick is that we can read
    compound_order() without any protection, if we are careful to filter out
    values larger than MAX_ORDER.  The only danger is that we skip too much.
    The same trick was already used for reading the freepage order in the
    migrate scanner.
    
    To demonstrate improvements of Patches 4 and 5 I've run stress-highalloc
    from mmtests, set to simulate THP allocations (including __GFP_COMP) on
    a 4GB system where 1GB was occupied by hugetlbfs pages.  I'll include
    just the relevant stats:
    
                                   Patch 3     Patch 4     Patch 5
    
    Compaction stalls                 7523        7529        7515
    Compaction success                 323         304         322
    Compaction failures               7200        7224        7192
    Page migrate success            247778      264395      240737
    Page migrate failure             15358       33184       21621
    Compaction pages isolated       906928      980192      909983
    Compaction migrate scanned     2005277     1692805     1498800
    Compaction free scanned       13255284    11539986     9011276
    Compaction cost                    288         305         277
    
    With 5 iterations per patch, the results are still noisy, but we can see
    that Patch 4 does reduce migrate_scanned by 15% thanks to skipping the
    hugetlbfs pages at once.  Interestingly, free_scanned is also reduced
    and I have no idea why.  Patch 5 further reduces free_scanned as
    expected, by 15%.  Other stats are unaffected modulo noise.
    
    [1] https://lkml.org/lkml/2015/1/19/158
    
    This patch (of 5):
    
    Compaction should finish when the migration and free scanner meet, i.e.
    they reach the same pageblock.  Currently however, the test in
    compact_finished() simply just compares the exact pfns, which may yield
    a false negative when the free scanner position is in the middle of a
    pageblock and the migration scanner reaches the begining of the same
    pageblock.
    
    This hasn't been a problem until commit e14c720e ("mm, compaction:
    remember position within pageblock in free pages scanner") allowed the
    free scanner position to be in the middle of a pageblock between
    invocations.  The hot-fix 1d5bfe1f ("mm, compaction: prevent
    infinite loop in compact_zone") prevented the issue by adding a special
    check in the migration scanner to satisfy the current detection of
    scanners meeting.
    
    However, the proper fix is to make the detection more robust.  This
    patch introduces the compact_scanners_met() function that returns true
    when the free scanner position is in the same or lower pageblock than
    the migration scanner.  The special case in isolate_migratepages()
    introduced by 1d5bfe1f is removed.
    Suggested-by: default avatarJoonsoo Kim <iamjoonsoo.kim@lge.com>
    Signed-off-by: default avatarVlastimil Babka <vbabka@suse.cz>
    Cc: Minchan Kim <minchan@kernel.org>
    Acked-by: default avatarMel Gorman <mgorman@suse.de>
    Acked-by: default avatarJoonsoo Kim <iamjoonsoo.kim@lge.com>
    Acked-by: default avatarMichal Nazarewicz <mina86@mina86.com>
    Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
    Cc: Christoph Lameter <cl@linux.com>
    Acked-by: default avatarRik van Riel <riel@redhat.com>
    Cc: David Rientjes <rientjes@google.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    f2849aa0
compaction.c 47.4 KB