• Hou Tao's avatar
    bpf, arm64: Support more atomic operations · 1902472b
    Hou Tao authored
    Atomics for eBPF patch series adds support for atomic[64]_fetch_add,
    atomic[64]_[fetch_]{and,or,xor} and atomic[64]_{xchg|cmpxchg}, but it
    only adds support for x86-64, so support these atomic operations for
    arm64 as well.
    
    Basically the implementation procedure is almost mechanical translation
    of code snippets in atomic_ll_sc.h & atomic_lse.h & cmpxchg.h located
    under arch/arm64/include/asm.
    
    When LSE atomic is unavailable, an extra temporary register is needed for
    (BPF_ADD | BPF_FETCH) to save the value of src register, instead of adding
    TMP_REG_4 just use BPF_REG_AX instead. Also make emit_lse_atomic() as an
    empty inline function when CONFIG_ARM64_LSE_ATOMICS is disabled.
    
    For cpus_have_cap(ARM64_HAS_LSE_ATOMICS) case and no-LSE-ATOMICS case, the
    following three tests: "./test_verifier", "./test_progs -t atomic" and
    "insmod ./test_bpf.ko" are exercised and passed.
    Signed-off-by: default avatarHou Tao <houtao1@huawei.com>
    Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
    Link: https://lore.kernel.org/bpf/20220217072232.1186625-4-houtao1@huawei.com
    1902472b
bpf_jit_comp.c 33.6 KB