• Vasily Gorbik's avatar
    s390/ftrace: Avoid calling unwinder in ftrace_return_address() · a84dd0d8
    Vasily Gorbik authored
    ftrace_return_address() is called extremely often from
    performance-critical code paths when debugging features like
    CONFIG_TRACE_IRQFLAGS are enabled. For example, with debug_defconfig,
    ftrace selftests on my LPAR currently execute ftrace_return_address()
    as follows:
    
    ftrace_return_address(0) - 0 times (common code uses __builtin_return_address(0) instead)
    ftrace_return_address(1) - 2,986,805,401 times (with this patch applied)
    ftrace_return_address(2) - 140 times
    ftrace_return_address(>2) - 0 times
    
    The use of __builtin_return_address(n) was replaced by return_address()
    with an unwinder call by commit cae74ba8 ("s390/ftrace:
    Use unwinder instead of __builtin_return_address()") because
    __builtin_return_address(n) simply walks the stack backchain and doesn't
    check for reaching the stack top. For shallow stacks with fewer than
    "n" frames, this results in reads at low addresses and random
    memory accesses.
    
    While calling the fully functional unwinder "works", it is very slow
    for this purpose. Moreover, potentially following stack switches and
    walking past IRQ context is simply wrong thing to do for
    ftrace_return_address().
    
    Reimplement return_address() to essentially be __builtin_return_address(n)
    with checks for reaching the stack top. Since the ftrace_return_address(n)
    argument is always a constant, keep the implementation in the header,
    allowing both GCC and Clang to unroll the loop and optimize it to the
    bare minimum.
    
    Fixes: cae74ba8 ("s390/ftrace: Use unwinder instead of __builtin_return_address()")
    Cc: stable@vger.kernel.org
    Reported-by: default avatarSumanth Korikkar <sumanthk@linux.ibm.com>
    Reviewed-by: default avatarHeiko Carstens <hca@linux.ibm.com>
    Acked-by: default avatarSumanth Korikkar <sumanthk@linux.ibm.com>
    Signed-off-by: default avatarVasily Gorbik <gor@linux.ibm.com>
    a84dd0d8
ftrace.h 4.52 KB