• Josh Poimboeuf's avatar
    bpf: Disable GCC -fgcse optimization for ___bpf_prog_run() · 3193c083
    Josh Poimboeuf authored
    On x86-64, with CONFIG_RETPOLINE=n, GCC's "global common subexpression
    elimination" optimization results in ___bpf_prog_run()'s jumptable code
    changing from this:
    
    	select_insn:
    		jmp *jumptable(, %rax, 8)
    		...
    	ALU64_ADD_X:
    		...
    		jmp *jumptable(, %rax, 8)
    	ALU_ADD_X:
    		...
    		jmp *jumptable(, %rax, 8)
    
    to this:
    
    	select_insn:
    		mov jumptable, %r12
    		jmp *(%r12, %rax, 8)
    		...
    	ALU64_ADD_X:
    		...
    		jmp *(%r12, %rax, 8)
    	ALU_ADD_X:
    		...
    		jmp *(%r12, %rax, 8)
    
    The jumptable address is placed in a register once, at the beginning of
    the function.  The function execution can then go through multiple
    indirect jumps which rely on that same register value.  This has a few
    issues:
    
    1) Objtool isn't smart enough to be able to track such a register value
       across multiple recursive indirect jumps through the jump table.
    
    2) With CONFIG_RETPOLINE enabled, this optimization actually results in
       a small slowdown.  I measured a ~4.7% slowdown in the test_bpf
       "tcpdump port 22" selftest.
    
       This slowdown is actually predicted by the GCC manual:
    
         Note: When compiling a program using computed gotos, a GCC
         extension, you may get better run-time performance if you
         disable the global common subexpression elimination pass by
         adding -fno-gcse to the command line.
    
    So just disable the optimization for this function.
    
    Fixes: e55a7325 ("bpf: Fix ORC unwinding in non-JIT BPF code")
    Reported-by: default avatarRandy Dunlap <rdunlap@infradead.org>
    Signed-off-by: default avatarJosh Poimboeuf <jpoimboe@redhat.com>
    Signed-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
    Acked-by: default avatarAlexei Starovoitov <ast@kernel.org>
    Acked-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
    Link: https://lkml.kernel.org/r/30c3ca29ba037afcbd860a8672eef0021addf9fe.1563413318.git.jpoimboe@redhat.com
    3193c083
core.c 54 KB