- 12 Feb, 2021 11 commits
-
-
Martin KaFai Lau authored
This patch adds a "void *owner" member. The existing bpf_tcp_ca test will ensure the bpf_cubic.o and bpf_dctcp.o can be loaded. Signed-off-by:
Martin KaFai Lau <kafai@fb.com> Signed-off-by:
Andrii Nakryiko <andrii@kernel.org> Acked-by:
-
Martin KaFai Lau authored
When libbpf initializes the kernel's struct_ops in "bpf_map__init_kern_struct_ops()", it enforces all pointer types must be a function pointer and rejects others. It turns out to be too strict. For example, when directly using "struct tcp_congestion_ops" from vmlinux.h, it has a "struct module *owner" member and it is set to NULL in a bpf_tcp_cc.o. Instead, it only needs to ensure the member is a function pointer if it has been set (relocated) to a bpf-prog. This patch moves the "btf_is_func_proto(kern_mtype)" check after the existing "if (!prog) { continue; }". The original debug message in "if (!prog) { continue; }" is also removed since it is no longer valid. Beside, there is a later debug message to tell which function pointer is set. The "btf_is_func_proto(mtype)" has already been guaranteed in "bpf_object__collect_st_ops_relos()" which has been run before "bpf_map__init_kern_struct_ops()". Thus, this check is removed. v2: - Remove outdated debug message (Andrii) Remove because there is a later debug message to tell which function pointer is set. - Following mtype->type is no longer needed. Remove: "skip_mods_and_typedefs(btf, mtype->type, &mtype_id)" - Do "if (!prog)" test before skip_mods_and_typedefs. Fixes: 590a0088 ("bpf: libbpf: Add STRUCT_OPS support") Signed-off-by: -
Stanislav Fomichev authored
We have the environments where usage of AF_INET is prohibited (cgroup/sock_create returns EPERM for AF_INET). Let's use AF_LOCAL instead of AF_INET, it should perfectly work with SIOCETHTOOL. Signed-off-by:
Stanislav Fomichev <sdf@google.com> Signed-off-by:
Björn Töpel <bjorn.topel@intel.com> Acked-by:
-
Ilya Leoshkevich authored
All 32-bit variants of BPF_FETCH (add, and, or, xor, xchg, cmpxchg) define a 32-bit subreg and thus have zext_dst set. Their encoding, however, uses dst_reg field as a base register, which causes opt_subreg_zext_lo32_rnd_hi32() to zero-extend said base register instead of the one the insn really defines (r0 or src_reg). Fix by properly choosing a register being defined, similar to how check_atomic() already does that. Signed-off-by:
Ilya Leoshkevich <iii@linux.ibm.com> Signed-off-by:
Alexei Starovoitov <ast@kernel.org> Link: https://lore.kernel.org/bpf/20210210204502.83429-1-iii@linux.ibm.com
-
Ilya Leoshkevich authored
Based on [1], BPF_CMPXCHG should always load the old value into R0. The phrasing in bpf.rst is somewhat ambiguous in this regard, improve it to make this aspect crystal clear. [1] https://lore.kernel.org/bpf/CAADnVQJFcFwxEz=wnV=hkie-EDwa8s5JGbBQeFt1TGux1OihJw@mail.gmail.com/Signed-off-by:
-
Alexei Starovoitov authored
bpf_prog_realloc copies contents of struct bpf_prog. The pointers have to be cleared before freeing old struct. Reported-by:
-
Florent Revest authored
This builds up on the existing socket cookie test which checks whether the bpf_get_socket_cookie helpers provide the same value in cgroup/connect6 and sockops programs for a socket created by the userspace part of the test. Instead of having an update_cookie sockops program tag a socket local storage with 0xFF, this uses both an update_cookie_sockops program and an update_cookie_tracing program which succesively tag the socket with 0x0F and then 0xF0. Signed-off-by:
Florent Revest <revest@chromium.org> Signed-off-by:
KP Singh <kpsingh@kernel.org> Link: https://lore.kernel.org/bpf/20210210111406.785541-5-revest@chromium.org
-
Florent Revest authored
When migrating from the bpf.h's to the vmlinux.h's definition of struct bps_sock, an interesting LLVM behavior happened. LLVM started producing two fetches of ctx->sk in the sockops program this means that the verifier could not keep track of the NULL-check on ctx->sk. Therefore, we need to extract ctx->sk in a variable before checking and dereferencing it. Signed-off-by:
-
Florent Revest authored
Currently, the selftest for the BPF socket_cookie helpers is built and run independently from test_progs. It's easy to forget and hard to maintain. This patch moves the socket cookies test into prog_tests/ and vastly simplifies its logic by: - rewriting the loading code with BPF skeletons - rewriting the server/client code with network helpers - rewriting the cgroup code with test__join_cgroup - rewriting the error handling code with CHECKs Signed-off-by:
-
Florent Revest authored
This needs a new helper that: - can work in a sleepable context (using sock_gen_cookie) - takes a struct sock pointer and checks that it's not NULL Signed-off-by:
-
Florent Revest authored
Since "92acdc58 bpf, net: Rework cookie generator as per-cpu one" socket cookies are not guaranteed to be non-decreasing. The bpf_get_socket_cookie helper descriptions are currently specifying that cookies are non-decreasing but we don't want users to rely on that. Reported-by:
Daniel Borkmann <daniel@iogearbox.net> Signed-off-by:
-
- 11 Feb, 2021 10 commits
-
-
Jiri Olsa authored
Nathan reported issue with cleaning empty build directory: $ make -s O=build distclean ../../scripts/Makefile.include:4: *** \ O=/ho...build/tools/bpf/resolve_btfids does not exist. Stop. The problem that tools scripts require existing output directory, otherwise it fails. Adding check around the resolve_btfids clean target to ensure the output directory is in place. Signed-off-by:
Jiri Olsa <jolsa@kernel.org> Signed-off-by:
Nathan Chancellor <nathan@kernel.org> Link: https://lore.kernel.org/bpf/20210211124004.1144344-1-jolsa@kernel.org
-
Alexei Starovoitov authored
Add a basic test for map-in-map and per-cpu maps in sleepable programs. Signed-off-by:
-
Alexei Starovoitov authored
Since sleepable programs are now executing under migrate_disable the per-cpu maps are safe to use. The map-in-map were ok to use in sleepable from the time sleepable progs were introduced. Note that non-preallocated maps are still not safe, since there is no rcu_read_lock yet in sleepable programs and dynamically allocated map elements are relying on rcu protection. The sleepable programs have rcu_read_lock_trace instead. That limitation will be addresses in the future. Signed-off-by:
-
Alexei Starovoitov authored
Since recursion_misses counter is available in bpf_prog_info improve the selftest to make sure it's counting correctly. Signed-off-by:
-
Alexei Starovoitov authored
Add per-program counter for number of times recursion prevention mechanism was triggered and expose it via show_fdinfo and bpf_prog_info. Teach bpftool to print it. Signed-off-by:
-
Alexei Starovoitov authored
Add recursive non-sleepable fentry program as a test. All attach points where sleepable progs can execute are non recursive so far. The recursion protection mechanism for sleepable cannot be activated yet. Signed-off-by:
-
Alexei Starovoitov authored
Since both sleepable and non-sleepable programs execute under migrate_disable add recursion prevention mechanism to both types of programs when they're executed via bpf trampoline. Signed-off-by:
-
Alexei Starovoitov authored
Since sleepable programs don't migrate from the cpu the excution stats can be computed for them as well. Reuse the same infrastructure for both sleepable and non-sleepable programs. run_cnt -> the number of times the program was executed. run_time_ns -> the program execution time in nanoseconds including the off-cpu time when the program was sleeping. Signed-off-by: -
Alexei Starovoitov authored
In older non-RT kernels migrate_disable() was the same as preempt_disable(). Since commit 74d862b6 ("sched: Make migrate_disable/enable() independent of RT") migrate_disable() is real and doesn't prevent sleeping. Running sleepable programs with migration disabled allows to add support for program stats and per-cpu maps later. Signed-off-by:
-
Alexei Starovoitov authored
Move bpf_prog_stats from prog->aux into prog to avoid one extra load in critical path of program execution. Signed-off-by:
-
- 10 Feb, 2021 8 commits
-
-
Marco Elver authored
For double-checked locking in bpf_common_lru_push_free(), node->type is read outside the critical section and then re-checked under the lock. However, concurrent writes to node->type result in data races. For example, the following concurrent access was observed by KCSAN: write to 0xffff88801521bc22 of 1 bytes by task 10038 on cpu 1: __bpf_lru_node_move_in kernel/bpf/bpf_lru_list.c:91 __local_list_flush kernel/bpf/bpf_lru_list.c:298 ... read to 0xffff88801521bc22 of 1 bytes by task 10043 on cpu 0: bpf_common_lru_push_free kernel/bpf/bpf_lru_list.c:507 bpf_lru_push_free kernel/bpf/bpf_lru_list.c:555 ... Fix the data races where node->type is read outside the critical section (for double-checked locking) by marking the access with READ_ONCE() as well as ensuring the variable is only accessed once. Fixes: 3a08c2fd ("bpf: LRU List") Reported-by: syzbot+3536db46dfa58c573458@syzkaller.appspotmail.com Reported-by: syzbot+516acdb03d3e27d91bcd@syzkaller.appspotmail.com Signed-off-by:
Marco Elver <elver@google.com> Signed-off-by:
-
Jiapeng Chong authored
Fix the following coccicheck warnings: ./tools/testing/selftests/bpf/xdpxceiver.c:954:28-30: WARNING !A || A && B is equivalent to !A || B. ./tools/testing/selftests/bpf/xdpxceiver.c:932:28-30: WARNING !A || A && B is equivalent to !A || B. ./tools/testing/selftests/bpf/xdpxceiver.c:909:28-30: WARNING !A || A && B is equivalent to !A || B. Reported-by:
Abaci Robot <abaci@linux.alibaba.com> Signed-off-by:
Jiapeng Chong <jiapeng.chong@linux.alibaba.com> Signed-off-by:
-
Ilya Leoshkevich authored
Atomic tests store a DW, but then load it back as a W from the same address. This doesn't work on big-endian systems, and since the point of those tests is not testing narrow loads, fix simply by loading a DW. Fixes: 98d666d0 ("bpf: Add tests for new BPF atomic operations") Signed-off-by:
-
Alexei Starovoitov authored
Andrei Matei says: ==================== Before this patch, variable offset access to the stack was dissalowed for regular instructions, but was allowed for "indirect" accesses (i.e. helpers). This patch removes the restriction, allowing reading and writing to the stack through stack pointers with variable offsets. This makes stack-allocated buffers more usable in programs, and brings stack pointers closer to other types of pointers. The motivation is being able to use stack-allocated buffers for data manipulation. When the stack size limit is sufficient, allocating buffers on the stack is simpler than per-cpu arrays, or other alternatives. V2 -> V3 - var-offset writes mark all the stack slots in range as initialized, so that future reads are not rejected. - rewrote the C test to not use uprobes, as per Andrii's suggestion. - addressed other review comments from Alexei. V1 -> V2 - add support for var-offset stack writes, in addition to reads - add a C test - made variable offset direct reads no longer destroy spilled registers in the access range - address review nits ==================== Signed-off-by: -
Andrei Matei authored
Add a higher-level test (C BPF program) for the new functionality - variable access stack reads and writes. Signed-off-by:
Andrei Matei <andreimatei1@gmail.com> Signed-off-by:
-
Andrei Matei authored
Add tests for the new functionality - reading and writing to the stack through a variable-offset pointer. Signed-off-by:
-
Andrei Matei authored
The verifier errors around stack accesses have changed slightly in the previous commit (generally for the better). Signed-off-by:
-
Andrei Matei authored
Before this patch, variable offset access to the stack was dissalowed for regular instructions, but was allowed for "indirect" accesses (i.e. helpers). This patch removes the restriction, allowing reading and writing to the stack through stack pointers with variable offsets. This makes stack-allocated buffers more usable in programs, and brings stack pointers closer to other types of pointers. The motivation is being able to use stack-allocated buffers for data manipulation. When the stack size limit is sufficient, allocating buffers on the stack is simpler than per-cpu arrays, or other alternatives. In unpriviledged programs, variable-offset reads and writes are disallowed (they were already disallowed for the indirect access case) because the speculative execution checking code doesn't support them. Additionally, when writing through a variable-offset stack pointer, if any pointers are in the accessible range, there's possilibities of later leaking pointers because the write cannot be tracked precisely. Writes with variable offset mark the whole range as initialized, even though we don't know which stack slots are actually written. This is in order to not reject future reads to these slots. Note that this doesn't affect writes done through helpers; like before, helpers need the whole stack range to be initialized to begin with. All the stack slots are in range are considered scalars after the write; variable-offset register spills are not tracked. For reads, all the stack slots in the variable range needs to be initialized (but see above about what writes do), otherwise the read is rejected. All register spilled in stack slots that might be read are marked as having been read, however reads through such pointers don't do register filling; the target register will always be either a scalar or a constant zero. Signed-off-by:
-
- 09 Feb, 2021 8 commits
-
-
Andrii Nakryiko authored
Jiri Olsa says: ==================== hi, resolve_btfids tool is used during the kernel build, so we should clean it on kernel's make clean. v2 changes: - add Song's acks on patches 1 and 4 (others changed) [Song] - add missing / [Andrii] - change srctree variable initialization [Andrii] - shifted ifdef for clean target [Andrii] thanks, jirka ==================== Signed-off-by: -
Jiri Olsa authored
The resolve_btfids tool is used during the kernel build, so we should clean it on kernel's make clean. Invoking the the resolve_btfids clean as part of root 'make clean'. Signed-off-by:
Song Liu <songliubraving@fb.com> Link: https://lore.kernel.org/bpf/20210205124020.683286-5-jolsa@kernel.org
-
Jiri Olsa authored
We want this clean to be called from tree's root Makefile, which defines same srctree variable and that will screw the make setup. We actually do not use srctree being passed from outside, so we can solve this by setting current srctree value directly. Also changing the way how srctree is initialized as suggested by Andrri. Also root Makefile does not define the implicit RM variable, so adding RM initialization. Signed-off-by:
-
Jiri Olsa authored
We want this clean to be called from tree's root clean and that one is silent if there's nothing to clean. Adding check for all object to clean and display CLEAN messages only if there are objects to remove. Signed-off-by:
-
Jiri Olsa authored
Setting up separate build directories for libbpf and libpsubcmd, so it's separated from other objects and we don't get them mixed in the future. It also simplifies cleaning, which is now simple rm -rf. Also there's no need for FEATURE-DUMP.libbpf and bpf_helper_defs.h files in .gitignore anymore. Signed-off-by:
-
Jiapeng Chong authored
Fix the following coccicheck warning: ./tools/bpf/bpf_dbg.c:893:32-36: WARNING: Comparison to bool. Reported-by:
-
Brendan Jackman authored
Add missing skeleton destroy call. Fixes: 37086bfd ("bpf: Propagate stack bounds to registers in atomics w/ BPF_FETCH") Reported-by:
Yonghong Song <yhs@fb.com> Signed-off-by:
Brendan Jackman <jackmanb@google.com> Signed-off-by:
-
Yang Li authored
Eliminate the following coccicheck warning: ./tools/testing/selftests/bpf/test_flow_dissector.c:506:2-3: Unneeded semicolon Reported-by:
Yang Li <yang.lee@linux.alibaba.com> Signed-off-by:
-
- 08 Feb, 2021 1 commit
-
-
Yang Li authored
Eliminate the following coccicheck warning: ./tools/testing/selftests/bpf/benchs/bench_ringbufs.c:322:2-3: Unneeded semicolon Reported-by:
-
- 05 Feb, 2021 2 commits
-
-
Yonghong Song authored
There is no functionality change. This refactoring intends to facilitate next patch change with BPF_PSEUDO_FUNC. Signed-off-by:
-
Andrii Nakryiko authored
KP Singh says: ==================== - Use ring_buffer__consume without BPF_RB_FORCE_WAKEUP as suggested by Andrii - Use ASSERT_OK_PTR macro Sleepable programs currently do not have access to any ringbuffer and since the perf ring buffer is a per-cpu map, it would not be trivial to enable for sleepable programs. Our specific use-case is to use the bpf_ima_inode_hash helper and write the hash to a ring buffer from a sleepable LSM hook. This series allows the BPF ringbuffer to be used in sleepable programs (tracing and lsm). Since the helper prototypes were already exposed the only change required was have the verifier allow BPF_MAP_TYPE_RINGBUF for sleepable programs. The ima test is also modified to use the ringbuffer instead of global variables. Based on dicussions we had over the BPF office hours and enabling all the possible debug options, I could not find any issues or warnings when using the ring buffer from sleepable programs. ==================== Signed-off-by:
-
