• Mel Gorman's avatar
    mm, page_alloc: use new PageAnonHead helper in the free page fast path · 17514574
    Mel Gorman authored
    The PageAnon check always checks for compound_head but this is a
    relatively expensive check if the caller already knows the page is a
    head page.  This patch creates a helper and uses it in the page free
    path which only operates on head pages.
    
    With this patch and "Only check PageCompound for high-order pages", the
    performance difference on a page allocator microbenchmark is;
    
                                                 4.6.0-rc2                  4.6.0-rc2
                                                   vanilla           nocompound-v1r20
      Min      alloc-odr0-1               425.00 (  0.00%)           417.00 (  1.88%)
      Min      alloc-odr0-2               313.00 (  0.00%)           308.00 (  1.60%)
      Min      alloc-odr0-4               257.00 (  0.00%)           253.00 (  1.56%)
      Min      alloc-odr0-8               224.00 (  0.00%)           221.00 (  1.34%)
      Min      alloc-odr0-16              208.00 (  0.00%)           205.00 (  1.44%)
      Min      alloc-odr0-32              199.00 (  0.00%)           199.00 (  0.00%)
      Min      alloc-odr0-64              195.00 (  0.00%)           193.00 (  1.03%)
      Min      alloc-odr0-128             192.00 (  0.00%)           191.00 (  0.52%)
      Min      alloc-odr0-256             204.00 (  0.00%)           200.00 (  1.96%)
      Min      alloc-odr0-512             213.00 (  0.00%)           212.00 (  0.47%)
      Min      alloc-odr0-1024            219.00 (  0.00%)           219.00 (  0.00%)
      Min      alloc-odr0-2048            225.00 (  0.00%)           225.00 (  0.00%)
      Min      alloc-odr0-4096            230.00 (  0.00%)           231.00 ( -0.43%)
      Min      alloc-odr0-8192            235.00 (  0.00%)           234.00 (  0.43%)
      Min      alloc-odr0-16384           235.00 (  0.00%)           234.00 (  0.43%)
      Min      free-odr0-1                215.00 (  0.00%)           191.00 ( 11.16%)
      Min      free-odr0-2                152.00 (  0.00%)           136.00 ( 10.53%)
      Min      free-odr0-4                119.00 (  0.00%)           107.00 ( 10.08%)
      Min      free-odr0-8                106.00 (  0.00%)            96.00 (  9.43%)
      Min      free-odr0-16                97.00 (  0.00%)            87.00 ( 10.31%)
      Min      free-odr0-32                91.00 (  0.00%)            83.00 (  8.79%)
      Min      free-odr0-64                89.00 (  0.00%)            81.00 (  8.99%)
      Min      free-odr0-128               88.00 (  0.00%)            80.00 (  9.09%)
      Min      free-odr0-256              106.00 (  0.00%)            95.00 ( 10.38%)
      Min      free-odr0-512              116.00 (  0.00%)           111.00 (  4.31%)
      Min      free-odr0-1024             125.00 (  0.00%)           118.00 (  5.60%)
      Min      free-odr0-2048             133.00 (  0.00%)           126.00 (  5.26%)
      Min      free-odr0-4096             136.00 (  0.00%)           130.00 (  4.41%)
      Min      free-odr0-8192             138.00 (  0.00%)           130.00 (  5.80%)
      Min      free-odr0-16384            137.00 (  0.00%)           130.00 (  5.11%)
    
    There is a sizable boost to the free allocator performance.  While there
    is an apparent boost on the allocation side, it's likely a co-incidence
    or due to the patches slightly reducing cache footprint.
    Signed-off-by: default avatarMel Gorman <mgorman@techsingularity.net>
    Acked-by: default avatarVlastimil Babka <vbabka@suse.cz>
    Cc: Jesper Dangaard Brouer <brouer@redhat.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    17514574
page_alloc.c 198 KB