- 11 Feb, 2021 8 commits
-
-
Alexei Starovoitov authored
Since sleepable programs are now executing under migrate_disable the per-cpu maps are safe to use. The map-in-map were ok to use in sleepable from the time sleepable progs were introduced. Note that non-preallocated maps are still not safe, since there is no rcu_read_lock yet in sleepable programs and dynamically allocated map elements are relying on rcu protection. The sleepable programs have rcu_read_lock_trace instead. That limitation will be addresses in the future. Signed-off-by:
Alexei Starovoitov <ast@kernel.org> Signed-off-by:
Daniel Borkmann <daniel@iogearbox.net> Acked-by:
Andrii Nakryiko <andrii@kernel.org> Acked-by:
KP Singh <kpsingh@kernel.org> Link: https://lore.kernel.org/bpf/20210210033634.62081-9-alexei.starovoitov@gmail.com
-
Alexei Starovoitov authored
Since recursion_misses counter is available in bpf_prog_info improve the selftest to make sure it's counting correctly. Signed-off-by:
-
Alexei Starovoitov authored
Add per-program counter for number of times recursion prevention mechanism was triggered and expose it via show_fdinfo and bpf_prog_info. Teach bpftool to print it. Signed-off-by:
-
Alexei Starovoitov authored
Add recursive non-sleepable fentry program as a test. All attach points where sleepable progs can execute are non recursive so far. The recursion protection mechanism for sleepable cannot be activated yet. Signed-off-by:
-
Alexei Starovoitov authored
Since both sleepable and non-sleepable programs execute under migrate_disable add recursion prevention mechanism to both types of programs when they're executed via bpf trampoline. Signed-off-by:
-
Alexei Starovoitov authored
Since sleepable programs don't migrate from the cpu the excution stats can be computed for them as well. Reuse the same infrastructure for both sleepable and non-sleepable programs. run_cnt -> the number of times the program was executed. run_time_ns -> the program execution time in nanoseconds including the off-cpu time when the program was sleeping. Signed-off-by: -
Alexei Starovoitov authored
In older non-RT kernels migrate_disable() was the same as preempt_disable(). Since commit 74d862b6 ("sched: Make migrate_disable/enable() independent of RT") migrate_disable() is real and doesn't prevent sleeping. Running sleepable programs with migration disabled allows to add support for program stats and per-cpu maps later. Signed-off-by:
-
Alexei Starovoitov authored
Move bpf_prog_stats from prog->aux into prog to avoid one extra load in critical path of program execution. Signed-off-by:
-
- 10 Feb, 2021 8 commits
-
-
Marco Elver authored
For double-checked locking in bpf_common_lru_push_free(), node->type is read outside the critical section and then re-checked under the lock. However, concurrent writes to node->type result in data races. For example, the following concurrent access was observed by KCSAN: write to 0xffff88801521bc22 of 1 bytes by task 10038 on cpu 1: __bpf_lru_node_move_in kernel/bpf/bpf_lru_list.c:91 __local_list_flush kernel/bpf/bpf_lru_list.c:298 ... read to 0xffff88801521bc22 of 1 bytes by task 10043 on cpu 0: bpf_common_lru_push_free kernel/bpf/bpf_lru_list.c:507 bpf_lru_push_free kernel/bpf/bpf_lru_list.c:555 ... Fix the data races where node->type is read outside the critical section (for double-checked locking) by marking the access with READ_ONCE() as well as ensuring the variable is only accessed once. Fixes: 3a08c2fd ("bpf: LRU List") Reported-by: syzbot+3536db46dfa58c573458@syzkaller.appspotmail.com Reported-by: syzbot+516acdb03d3e27d91bcd@syzkaller.appspotmail.com Signed-off-by:
Marco Elver <elver@google.com> Signed-off-by:
Martin KaFai Lau <kafai@fb.com> Link: https://lore.kernel.org/bpf/20210209112701.3341724-1-elver@google.com
-
Jiapeng Chong authored
Fix the following coccicheck warnings: ./tools/testing/selftests/bpf/xdpxceiver.c:954:28-30: WARNING !A || A && B is equivalent to !A || B. ./tools/testing/selftests/bpf/xdpxceiver.c:932:28-30: WARNING !A || A && B is equivalent to !A || B. ./tools/testing/selftests/bpf/xdpxceiver.c:909:28-30: WARNING !A || A && B is equivalent to !A || B. Reported-by:
Abaci Robot <abaci@linux.alibaba.com> Signed-off-by:
Jiapeng Chong <jiapeng.chong@linux.alibaba.com> Signed-off-by:
-
Ilya Leoshkevich authored
Atomic tests store a DW, but then load it back as a W from the same address. This doesn't work on big-endian systems, and since the point of those tests is not testing narrow loads, fix simply by loading a DW. Fixes: 98d666d0 ("bpf: Add tests for new BPF atomic operations") Signed-off-by:
Ilya Leoshkevich <iii@linux.ibm.com> Signed-off-by:
-
Alexei Starovoitov authored
Andrei Matei says: ==================== Before this patch, variable offset access to the stack was dissalowed for regular instructions, but was allowed for "indirect" accesses (i.e. helpers). This patch removes the restriction, allowing reading and writing to the stack through stack pointers with variable offsets. This makes stack-allocated buffers more usable in programs, and brings stack pointers closer to other types of pointers. The motivation is being able to use stack-allocated buffers for data manipulation. When the stack size limit is sufficient, allocating buffers on the stack is simpler than per-cpu arrays, or other alternatives. V2 -> V3 - var-offset writes mark all the stack slots in range as initialized, so that future reads are not rejected. - rewrote the C test to not use uprobes, as per Andrii's suggestion. - addressed other review comments from Alexei. V1 -> V2 - add support for var-offset stack writes, in addition to reads - add a C test - made variable offset direct reads no longer destroy spilled registers in the access range - address review nits ==================== Signed-off-by: -
Andrei Matei authored
Add a higher-level test (C BPF program) for the new functionality - variable access stack reads and writes. Signed-off-by:
Andrei Matei <andreimatei1@gmail.com> Signed-off-by:
-
Andrei Matei authored
Add tests for the new functionality - reading and writing to the stack through a variable-offset pointer. Signed-off-by:
-
Andrei Matei authored
The verifier errors around stack accesses have changed slightly in the previous commit (generally for the better). Signed-off-by:
-
Andrei Matei authored
Before this patch, variable offset access to the stack was dissalowed for regular instructions, but was allowed for "indirect" accesses (i.e. helpers). This patch removes the restriction, allowing reading and writing to the stack through stack pointers with variable offsets. This makes stack-allocated buffers more usable in programs, and brings stack pointers closer to other types of pointers. The motivation is being able to use stack-allocated buffers for data manipulation. When the stack size limit is sufficient, allocating buffers on the stack is simpler than per-cpu arrays, or other alternatives. In unpriviledged programs, variable-offset reads and writes are disallowed (they were already disallowed for the indirect access case) because the speculative execution checking code doesn't support them. Additionally, when writing through a variable-offset stack pointer, if any pointers are in the accessible range, there's possilibities of later leaking pointers because the write cannot be tracked precisely. Writes with variable offset mark the whole range as initialized, even though we don't know which stack slots are actually written. This is in order to not reject future reads to these slots. Note that this doesn't affect writes done through helpers; like before, helpers need the whole stack range to be initialized to begin with. All the stack slots are in range are considered scalars after the write; variable-offset register spills are not tracked. For reads, all the stack slots in the variable range needs to be initialized (but see above about what writes do), otherwise the read is rejected. All register spilled in stack slots that might be read are marked as having been read, however reads through such pointers don't do register filling; the target register will always be either a scalar or a constant zero. Signed-off-by:
-
- 09 Feb, 2021 8 commits
-
-
Andrii Nakryiko authored
Jiri Olsa says: ==================== hi, resolve_btfids tool is used during the kernel build, so we should clean it on kernel's make clean. v2 changes: - add Song's acks on patches 1 and 4 (others changed) [Song] - add missing / [Andrii] - change srctree variable initialization [Andrii] - shifted ifdef for clean target [Andrii] thanks, jirka ==================== Signed-off-by: -
Jiri Olsa authored
The resolve_btfids tool is used during the kernel build, so we should clean it on kernel's make clean. Invoking the the resolve_btfids clean as part of root 'make clean'. Signed-off-by:
Jiri Olsa <jolsa@kernel.org> Signed-off-by:
Song Liu <songliubraving@fb.com> Link: https://lore.kernel.org/bpf/20210205124020.683286-5-jolsa@kernel.org
-
Jiri Olsa authored
We want this clean to be called from tree's root Makefile, which defines same srctree variable and that will screw the make setup. We actually do not use srctree being passed from outside, so we can solve this by setting current srctree value directly. Also changing the way how srctree is initialized as suggested by Andrri. Also root Makefile does not define the implicit RM variable, so adding RM initialization. Signed-off-by:
-
Jiri Olsa authored
We want this clean to be called from tree's root clean and that one is silent if there's nothing to clean. Adding check for all object to clean and display CLEAN messages only if there are objects to remove. Signed-off-by:
-
Jiri Olsa authored
Setting up separate build directories for libbpf and libpsubcmd, so it's separated from other objects and we don't get them mixed in the future. It also simplifies cleaning, which is now simple rm -rf. Also there's no need for FEATURE-DUMP.libbpf and bpf_helper_defs.h files in .gitignore anymore. Signed-off-by:
-
Jiapeng Chong authored
Fix the following coccicheck warning: ./tools/bpf/bpf_dbg.c:893:32-36: WARNING: Comparison to bool. Reported-by:
-
Brendan Jackman authored
Add missing skeleton destroy call. Fixes: 37086bfd ("bpf: Propagate stack bounds to registers in atomics w/ BPF_FETCH") Reported-by:
Yonghong Song <yhs@fb.com> Signed-off-by:
Brendan Jackman <jackmanb@google.com> Signed-off-by:
-
Yang Li authored
Eliminate the following coccicheck warning: ./tools/testing/selftests/bpf/test_flow_dissector.c:506:2-3: Unneeded semicolon Reported-by:
Yang Li <yang.lee@linux.alibaba.com> Signed-off-by:
-
- 08 Feb, 2021 1 commit
-
-
Yang Li authored
Eliminate the following coccicheck warning: ./tools/testing/selftests/bpf/benchs/bench_ringbufs.c:322:2-3: Unneeded semicolon Reported-by:
-
- 05 Feb, 2021 7 commits
-
-
Yonghong Song authored
There is no functionality change. This refactoring intends to facilitate next patch change with BPF_PSEUDO_FUNC. Signed-off-by:
-
Andrii Nakryiko authored
KP Singh says: ==================== - Use ring_buffer__consume without BPF_RB_FORCE_WAKEUP as suggested by Andrii - Use ASSERT_OK_PTR macro Sleepable programs currently do not have access to any ringbuffer and since the perf ring buffer is a per-cpu map, it would not be trivial to enable for sleepable programs. Our specific use-case is to use the bpf_ima_inode_hash helper and write the hash to a ring buffer from a sleepable LSM hook. This series allows the BPF ringbuffer to be used in sleepable programs (tracing and lsm). Since the helper prototypes were already exposed the only change required was have the verifier allow BPF_MAP_TYPE_RINGBUF for sleepable programs. The ima test is also modified to use the ringbuffer instead of global variables. Based on dicussions we had over the BPF office hours and enabling all the possible debug options, I could not find any issues or warnings when using the ring buffer from sleepable programs. ==================== Signed-off-by: -
KP Singh authored
Instead of using shared global variables between userspace and BPF, use the ring buffer to send the IMA hash on the BPF ring buffer. This helps in validating both IMA and the usage of the ringbuffer in sleepable programs. Signed-off-by:
-
KP Singh authored
The BPF ringbuffer map is pre-allocated and the implementation logic does not rely on disabling preemption or per-cpu data structures. Using the BPF ringbuffer sleepable LSM and tracing programs does not trigger any warnings with DEBUG_ATOMIC_SLEEP, DEBUG_PREEMPT, PROVE_RCU and PROVE_LOCKING and LOCKDEP enabled. This allows helpers like bpf_copy_from_user and bpf_ima_inode_hash to write to the BPF ring buffer from sleepable BPF programs. Signed-off-by:
-
Andrii Nakryiko authored
KP Singh says: ==================== # v4 -> v5 - Use %Y (modification time) instead of %W (creation time) of the local copy of the kernel config to check for newer upstream config. - Rename the script to vmtest.sh # v3 -> v4 - Fix logic for updating kernel config to not download the file if there are no upstream modifications and avoid extraneous kernel compilation as suggested by Andrii. - This also removes the need for the -k flag. # v2 -> v3 - Fixes to silence verbose commands - Fixed output buffering without being teed out - Fixed the clobbered error code of the script - Other fixes suggested by Andrii # v1 -> v2 - The script now compiles the kernel by default, and the -k option implies "keep the kernel" - Pointer to the script in the docs. - Some minor simplifications. Allow developers and contributors to understand if their changes would end up breaking the BPF CI and avoid the back and forth required for fixing the test cases in the CI environment. The se ==================== Signed-off-by: -
KP Singh authored
Add a short note to make contributors aware of the existence of the script. The documentation does not intentionally document all the options of the script to avoid mentioning it in two places (it's available in the usage / help message of the script). Signed-off-by:
-
KP Singh authored
The script runs the BPF selftests locally on the same kernel image as they would run post submit in the BPF continuous integration framework. The goal of the script is to allow contributors to run selftests locally in the same environment to check if their changes would end up breaking the BPF CI and reduce the back-and-forth between the maintainers and the developers. Signed-off-by:
Jiri Olsa <jolsa@redhat.com> Link: https://lore.kernel.org/bpf/20210204194544.3383814-2-kpsingh@kernel.org
-
- 04 Feb, 2021 3 commits
-
-
Alexei Starovoitov authored
PTR_TO_BTF_ID registers contain either kernel pointer or NULL. Emit the NULL check explicitly by JIT instead of going into do_user_addr_fault() on NULL deference. Signed-off-by:
-
Andrii Nakryiko authored
Libbpf's Makefile relies on Linux tools infrastructure's feature detection framework, but libbpf's needs are very modest: it detects the presence of libelf and libz, both of which are mandatory. So it doesn't benefit much from the framework, but pays significant costs in terms of maintainability and debugging experience, when something goes wrong. The other feature detector, testing for the presernce of minimal BPF API in system headers is long obsolete as well, providing no value. So stop using feature detection and just assume the presence of libelf and libz during build time. Worst case, user will get a clear and actionable linker error, e.g.: /usr/bin/ld: cannot find -lelf On the other hand, we completely bypass recurring issues various users reported over time with false negatives of feature detection (libelf or libz not being detected, while they are actually present in the system). Signed-off-by:
John Fastabend <john.fastabend@gmail.com> Acked-by:
Randy Dunlap <rdunlap@infradead.org> Link: https://lore.kernel.org/bpf/20210203203445.3356114-1-andrii@kernel.org
-
Lorenzo Bianconi authored
Split ndo_xdp_xmit and ndo_start_xmit use cases in veth_xdp_rcv routine in order to alloc skbs in bulk for XDP_PASS verdict. Introduce xdp_alloc_skb_bulk utility routine to alloc skb bulk list. The proposed approach has been tested in the following scenario: eth (ixgbe) --> XDP_REDIRECT --> veth0 --> (remote-ns) veth1 --> XDP_PASS XDP_REDIRECT: xdp_redirect_map bpf sample XDP_PASS: xdp_rxq_info bpf sample traffic generator: pkt_gen sending udp traffic on a remote device bpf-next master: ~3.64Mpps bpf-next + skb bulking allocation: ~3.79Mpps Signed-off-by:
Lorenzo Bianconi <lorenzo@kernel.org> Signed-off-by:
Toshiaki Makita <toshiaki.makita1@gmail.com> Acked-by:
Jesper Dangaard Brouer <brouer@redhat.com> Link: https://lore.kernel.org/bpf/a14a30d3c06fff24e13f836c733d80efc0bd6eb5.1611957532.git.lorenzo@kernel.org
-
- 03 Feb, 2021 4 commits
-
-
Andrei Matei authored
This patch adds support to verifier tests to check for a succession of verifier log messages on program load failure. This makes the errstr field work uniformly across REJECT and VERBOSE_ACCEPT checks. This patch also increases the maximum size of a message in the series of messages to test from 80 chars to 200 chars. This is in order to keep existing tests working, which sometimes test for messages larger than 80 chars (which was accepted in the REJECT case, when testing for a single message, but not in the VERBOSE_ACCEPT case, when testing for possibly multiple messages). And example of such a long, checked message is in bounds.c: "R1 has unknown scalar with mixed signed bounds, pointer arithmetic with it prohibited for !root" Signed-off-by:
-
Yang Li authored
Eliminate the following coccicheck warning: ./samples/bpf/cookie_uid_helper_example.c:316:3-4: Unneeded semicolon Reported-by:
-
KP Singh authored
Some compilers trigger a warning when tmp_dir_path is allocated with a fixed size of 64-bytes and used in the following snprintf: snprintf(tmp_exec_path, sizeof(tmp_exec_path), "%s/copy_of_rm", tmp_dir_path); warning: ‘/copy_of_rm’ directive output may be truncated writing 11 bytes into a region of size between 1 and 64 [-Wformat-truncation=] This is because it assumes that tmp_dir_path can be a maximum of 64 bytes long and, therefore, the end-result can get truncated. Fix it by not using a fixed size in the initialization of tmp_dir_path which allows the compiler to track actual size of the array better. Fixes: 2f94ac19 ("bpf: Update local storage test to check handling of null ptrs") Signed-off-by:
-
Brendan Jackman authored
When BPF_FETCH is set, atomic instructions load a value from memory into a register. The current verifier code first checks via check_mem_access whether we can access the memory, and then checks via check_reg_arg whether we can write into the register. For loads, check_reg_arg has the side-effect of marking the register's value as unkonwn, and check_mem_access has the side effect of propagating bounds from memory to the register. This currently only takes effect for stack memory. Therefore with the current order, bounds information is thrown away, but by simply reversing the order of check_reg_arg vs. check_mem_access, we can instead propagate bounds smartly. A simple test is added with an infinite loop that can only be proved unreachable if this propagation is present. This is implemented both with C and directly in test_verifier using assembly. Suggested-by:
-
- 02 Feb, 2021 1 commit
-
-
Tiezhu Yang authored
There exists many build errors when make M=samples/bpf on the Loongson platform. This issue is MIPS related, x86 compiles just fine. Here are some errors: CLANG-bpf samples/bpf/sockex2_kern.o In file included from samples/bpf/sockex2_kern.c:2: In file included from ./include/uapi/linux/in.h:24: In file included from ./include/linux/socket.h:8: In file included from ./include/linux/uio.h:8: In file included from ./include/linux/kernel.h:11: In file included from ./include/linux/bitops.h:32: In file included from ./arch/mips/include/asm/bitops.h:19: In file included from ./arch/mips/include/asm/barrier.h:11: ./arch/mips/include/asm/addrspace.h:13:10: fatal error: 'spaces.h' file not found ^~~~~~~~~~ 1 error generated. CLANG-bpf samples/bpf/sockex2_kern.o In file included from samples/bpf/sockex2_kern.c:2: In file included from ./include/uapi/linux/in.h:24: In file included from ./include/linux/socket.h:8: In file included from ./include/linux/uio.h:8: In file included from ./include/linux/kernel.h:11: In file included from ./include/linux/bitops.h:32: In file included from ./arch/mips/include/asm/bitops.h:22: In file included from ./arch/mips/include/asm/cpu-features.h:13: In file included from ./arch/mips/include/asm/cpu-info.h:15: In file included from ./include/linux/cache.h:6: ./arch/mips/include/asm/cache.h:12:10: fatal error: 'kmalloc.h' file not found ^~~~~~~~~~~ 1 error generated. CLANG-bpf samples/bpf/sockex2_kern.o In file included from samples/bpf/sockex2_kern.c:2: In file included from ./include/uapi/linux/in.h:24: In file included from ./include/linux/socket.h:8: In file included from ./include/linux/uio.h:8: In file included from ./include/linux/kernel.h:11: In file included from ./include/linux/bitops.h:32: In file included from ./arch/mips/include/asm/bitops.h:22: ./arch/mips/include/asm/cpu-features.h:15:10: fatal error: 'cpu-feature-overrides.h' file not found ^~~~~~~~~~~~~~~~~~~~~~~~~ 1 error generated. $ find arch/mips/include/asm -name spaces.h | sort arch/mips/include/asm/mach-ar7/spaces.h ... arch/mips/include/asm/mach-generic/spaces.h ... arch/mips/include/asm/mach-loongson64/spaces.h ... arch/mips/include/asm/mach-tx49xx/spaces.h $ find arch/mips/include/asm -name kmalloc.h | sort arch/mips/include/asm/mach-generic/kmalloc.h arch/mips/include/asm/mach-ip32/kmalloc.h arch/mips/include/asm/mach-tx49xx/kmalloc.h $ find arch/mips/include/asm -name cpu-feature-overrides.h | sort arch/mips/include/asm/mach-ath25/cpu-feature-overrides.h ... arch/mips/include/asm/mach-generic/cpu-feature-overrides.h ... arch/mips/include/asm/mach-loongson64/cpu-feature-overrides.h ... arch/mips/include/asm/mach-tx49xx/cpu-feature-overrides.h In the arch/mips/Makefile, there exists the following board-dependent options: include arch/mips/Kbuild.platforms cflags-y += -I$(srctree)/arch/mips/include/asm/mach-generic So we can do the similar things in samples/bpf/Makefile, just add platform specific and generic include dir for MIPS Loongson64 to fix the build errors. Signed-off-by:Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by:
-
