• Vlastimil Babka's avatar
    mm, rmap: handle anon_vma_prepare() common case inline · d5a187da
    Vlastimil Babka authored
    anon_vma_prepare() is mostly a large "if (unlikely(...))" block, as the
    expected common case is that an anon_vma already exists.  We could turn
    the condition around and return 0, but it also makes sense to do it
    inline and avoid a call for the common case.
    
    Bloat-o-meter naturally shows that inlining the check has some code size
    costs:
    
    add/remove: 1/1 grow/shrink: 4/0 up/down: 475/-373 (102)
    function                                     old     new   delta
    __anon_vma_prepare                             -     359    +359
    handle_mm_fault                             2744    2796     +52
    hugetlb_cow                                 1146    1170     +24
    hugetlb_fault                               2123    2145     +22
    wp_page_copy                                1469    1487     +18
    anon_vma_prepare                             373       -    -373
    
    Checking the asm however confirms that the hot paths now avoid a call,
    which is moved away.
    
    [akpm@linux-foundation.org: coding-style fixes]
    Link: http://lkml.kernel.org/r/20161116074005.22768-1-vbabka@suse.czSigned-off-by: default avatarVlastimil Babka <vbabka@suse.cz>
    Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
    Cc: Johannes Weiner <hannes@cmpxchg.org>
    Cc: Konstantin Khlebnikov <koct9i@gmail.com>
    Cc: Rik van Riel <riel@redhat.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    d5a187da
rmap.c 51.1 KB