- 01 May, 2020 6 commits
-
-
Andrii Nakryiko authored
If bpf_link_prime() succeeds to allocate new anon file, but then fails to allocate ID for it, link priming is considered to be failed and user is supposed ot be able to directly kfree() bpf_link, because it was never exposed to user-space. But at that point file already keeps a pointer to bpf_link and will eventually call bpf_link_release(), so if bpf_link was kfree()'d by caller, that would lead to use-after-free. Fix this by first allocating ID and only then allocating file. Adding ID to link_idr is ok, because link at that point still doesn't have its ID set, so no user-space process can create a new FD for it. Fixes: a3b80e10 ("bpf: Allocate ID for bpf_link") Reported-by: syzbot+39b64425f91b5aab714d@syzkaller.appspotmail.com Suggested-by:
Martin KaFai Lau <kafai@fb.com> Signed-off-by:
Andrii Nakryiko <andriin@fb.com> Signed-off-by:
Alexei Starovoitov <ast@kernel.org> Acked-by:
-
Stanislav Fomichev authored
Currently, bpf_getsockopt and bpf_setsockopt helpers operate on the 'struct bpf_sock_ops' context in BPF_PROG_TYPE_SOCK_OPS program. Let's generalize them and make them available for 'struct bpf_sock_addr'. That way, in the future, we can allow those helpers in more places. As an example, let's expose those 'struct bpf_sock_addr' based helpers to BPF_CGROUP_INET{4,6}_CONNECT hooks. That way we can override CC before the connection is made. v3: * Expose custom helpers for bpf_sock_addr context instead of doing generic bpf_sock argument (as suggested by Daniel). Even with try_socket_lock that doesn't sleep we have a problem where context sk is already locked and socket lock is non-nestable. v2: * s/BPF_PROG_TYPE_CGROUP_SOCKOPT/BPF_PROG_TYPE_SOCK_OPS/ Signed-off-by:Stanislav Fomichev <sdf@google.com> Signed-off-by:
John Fastabend <john.fastabend@gmail.com> Link: https://lore.kernel.org/bpf/20200430233152.199403-1-sdf@google.com
-
Alexei Starovoitov authored
Song Liu says: ==================== run_time_ns is a useful stats for BPF programs. However, it is gated by sysctl kernel.bpf_stats_enabled. When multiple user space tools are toggling kernl.bpf_stats_enabled at the same time, they may confuse each other. Solve this problem with a new BPF command BPF_ENABLE_STATS. Changes v8 => v9: 1. Clean up in selftest (Andrii). 2. Not using static variable in test program (Andrii). Changes v7 => v8: 1. Change name BPF_STATS_RUNTIME_CNT => BPF_STATS_RUN_TIME (Alexei). 2. Add CHECK_ATTR to bpf_enable_stats() (Alexei). 3. Rebase (Andrii). 4. Simplfy the selftest (Alexei). Changes v6 => v7: 1. Add test to verify run_cnt matches count measured by the program. Changes v5 => v6: 1. Simplify test program (Yonghong). 2. Rebase (with some conflicts). Changes v4 => v5: 1. Use memset to zero bpf_attr in bpf_enable_stats() (Andrii). Changes v3 => v4: 1. Add libbpf support and selftest; 2. Avoid cleaning trailing space. Changes v2 => v3: 1. Rename the command to BPF_ENABLE_STATS, and make it extendible. 2. fix commit log; 3. remove unnecessary headers. ==================== Signed-off-by: -
Song Liu authored
Add test for BPF_ENABLE_STATS, which should enable run_time_ns stats. ~/selftests/bpf# ./test_progs -t enable_stats -v test_enable_stats:PASS:skel_open_and_load 0 nsec test_enable_stats:PASS:get_stats_fd 0 nsec test_enable_stats:PASS:attach_raw_tp 0 nsec test_enable_stats:PASS:get_prog_info 0 nsec test_enable_stats:PASS:check_stats_enabled 0 nsec test_enable_stats:PASS:check_run_cnt_valid 0 nsec Summary: 1/0 PASSED, 0 SKIPPED, 0 FAILED Signed-off-by:
Song Liu <songliubraving@fb.com> Signed-off-by:
-
Song Liu authored
bpf_enable_stats() is added to enable given stats. Signed-off-by:
-
Song Liu authored
Currently, sysctl kernel.bpf_stats_enabled controls BPF runtime stats. Typical userspace tools use kernel.bpf_stats_enabled as follows: 1. Enable kernel.bpf_stats_enabled; 2. Check program run_time_ns; 3. Sleep for the monitoring period; 4. Check program run_time_ns again, calculate the difference; 5. Disable kernel.bpf_stats_enabled. The problem with this approach is that only one userspace tool can toggle this sysctl. If multiple tools toggle the sysctl at the same time, the measurement may be inaccurate. To fix this problem while keep backward compatibility, introduce a new bpf command BPF_ENABLE_STATS. On success, this command enables stats and returns a valid fd. BPF_ENABLE_STATS takes argument "type". Currently, only one type, BPF_STATS_RUN_TIME, is supported. We can extend the command to support other types of stats in the future. With BPF_ENABLE_STATS, user space tool would have the following flow: 1. Get a fd with BPF_ENABLE_STATS, and make sure it is valid; 2. Check program run_time_ns; 3. Sleep for the monitoring period; 4. Check program run_time_ns again, calculate the difference; 5. Close the fd. Signed-off-by:
-
- 30 Apr, 2020 3 commits
-
-
Jakub Sitnicki authored
Check that verifier allows passing a map of type: BPF_MAP_TYPE_REUSEPORT_SOCKARRARY, or BPF_MAP_TYPE_SOCKMAP, or BPF_MAP_TYPE_SOCKHASH ... to bpf_sk_select_reuseport helper. Suggested-by:
Jakub Sitnicki <jakub@cloudflare.com> Signed-off-by:
Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20200430104738.494180-1-jakub@cloudflare.com
-
Andrii Nakryiko authored
Some versions of GCC falsely detect that vi might not be initialized. That's not true, but let's silence it with NULL initialization. Signed-off-by:
-
Luke Nelson authored
This patch fixes issues with stackframe unwinding and alignment in the current stack layout for BPF programs on RV32. In the current layout, RV32 fp points to the JIT scratch registers, rather than to the callee-saved registers. This breaks stackframe unwinding, which expects fp to point just above the saved ra and fp registers. This patch fixes the issue by moving the callee-saved registers to be stored on the top of the stack, pointed to by fp. This satisfies the assumptions of stackframe unwinding. This patch also fixes an issue with the old layout that the stack was not aligned to 16 bytes. Stacktrace from JITed code using the old stack layout: [ 12.196249 ] [<c0402200>] walk_stackframe+0x0/0x96 Stacktrace using the new stack layout: [ 13.062888 ] [<c0402200>] walk_stackframe+0x0/0x96 [ 13.063028 ] [<c04023c6>] show_stack+0x28/0x32 [ 13.063253 ] [<a403e778>] bpf_prog_82b916b2dfa00464+0x80/0x908 [ 13.063417 ] [<c09270b2>] bpf_test_run+0x124/0x39a [ 13.063553 ] [<c09276c0>] bpf_prog_test_run_skb+0x234/0x448 [ 13.063704 ] [<c048510e>] __do_sys_bpf+0x766/0x13b4 [ 13.063840 ] [<c0485d82>] sys_bpf+0xc/0x14 [ 13.063961 ] [<c04010f0>] ret_from_syscall+0x0/0x2 The new code is also simpler to understand and includes an ASCII diagram of the stack layout. Tested on riscv32 QEMU virt machine. Signed-off-by:
Luke Nelson <luke.r.nels@gmail.com> Signed-off-by:
Xi Wang <xi.wang@gmail.com> Link: https://lore.kernel.org/bpf/20200430005127.2205-1-luke.r.nels@gmail.com
-
- 29 Apr, 2020 31 commits
-
-
Arnd Bergmann authored
Hiding the only using of bpf_link_type_strs[] in an #ifdef causes an unused-variable warning: kernel/bpf/syscall.c:2280:20: error: 'bpf_link_type_strs' defined but not used [-Werror=unused-variable] 2280 | static const char *bpf_link_type_strs[] = { Move the definition into the same #ifdef. Fixes: f2e10bff ("bpf: Add support for BPF_OBJ_GET_INFO_BY_FD for bpf_link") Signed-off-by:Arnd Bergmann <arnd@arndb.de> Signed-off-by:
-
Jakub Sitnicki authored
Update bpf_sk_assign test to fetch the server socket from SOCKMAP, now that map lookup from BPF in SOCKMAP is enabled. This way the test TC BPF program doesn't need to know what address server socket is bound to. Signed-off-by:
-
Jakub Sitnicki authored
Now that bpf_map_lookup_elem() is white-listed for SOCKMAP/SOCKHASH, replace the tests which check that verifier prevents lookup on these map types with ones that ensure that lookup operation is permitted, but only with a release of acquired socket reference. Signed-off-by:
-
Jakub Sitnicki authored
White-list map lookup for SOCKMAP/SOCKHASH from BPF. Lookup returns a pointer to a full socket and acquires a reference if necessary. To support it we need to extend the verifier to know that: (1) register storing the lookup result holds a pointer to socket, if lookup was done on SOCKMAP/SOCKHASH, and that (2) map lookup on SOCKMAP/SOCKHASH is a reference acquiring operation, which needs a corresponding reference release with bpf_sk_release. On sock_map side, lookup handlers exposed via bpf_map_ops now bump sk_refcnt if socket is reference counted. In turn, bpf_sk_select_reuseport, the only in-kernel user of SOCKMAP/SOCKHASH ops->map_lookup_elem, was updated to release the reference. Sockets fetched from a map can be used in the same way as ones returned by BPF socket lookup helpers, such as bpf_sk_lookup_tcp. In particular, they can be used with bpf_sk_assign to direct packets toward a socket on TC ingress path. Suggested-by:Lorenz Bauer <lmb@cloudflare.com> Signed-off-by:
-
Quentin Monnet authored
The new libcap dependency is not used for an essential feature of bpftool, and we could imagine building the tool without checks on CAP_SYS_ADMIN by disabling probing features as an unprivileged users. Make it so, in order to avoid a hard dependency on libcap, and to ease packaging/embedding of bpftool. Signed-off-by:
Quentin Monnet <quentin@isovalent.com> Signed-off-by:
-
Quentin Monnet authored
There is demand for a way to identify what BPF helper functions are available to unprivileged users. To do so, allow unprivileged users to run "bpftool feature probe" to list BPF-related features. This will only show features accessible to those users, and may not reflect the full list of features available (to administrators) on the system. To avoid the case where bpftool is inadvertently run as non-root and would list only a subset of the features supported by the system when it would be expected to list all of them, running as unprivileged is gated behind the "unprivileged" keyword passed to the command line. When used by a privileged user, this keyword allows to drop the CAP_SYS_ADMIN and to list the features available to unprivileged users. Note that this addsd a dependency on libpcap for compiling bpftool. Note that there is no particular reason why the probes were restricted to root, other than the fact I did not need them for unprivileged and did not bother with the additional checks at the time probes were added. Signed-off-by:
-
Quentin Monnet authored
The "full_mode" variable used for switching between full or partial feature probing (i.e. with or without probing helpers that will log warnings in kernel logs) was piped from the main do_probe() function down to probe_helpers_for_progtype(), where it is needed. Define it as a global variable: the calls will be more readable, and if other similar flags were to be used in the future, we could use global variables as well instead of extending again the list of arguments with new flags. Signed-off-by:
-
Alexei Starovoitov authored
Andrii Nakryiko says: ==================== Add necessary infra to build selftests with ASAN (or any other sanitizer). Fix a bunch of found memory leaks and other memory access issues. v1->v2: - don't add ASAN flavor, but allow extra flags for build (Alexei); - fix few more found issues, which somehow were missed first time. ==================== Signed-off-by: -
Andrii Nakryiko authored
With recent changes, runqslower is being copied into selftests/bpf root directory. So add it into .gitignore. Fixes: b26d1e2b ("selftests/bpf: Copy runqslower to OUTPUT directory") Signed-off-by:
-
Andrii Nakryiko authored
If condition is inverted, but it's also just not necessary. Fixes: 1c1052e0 ("tools/testing/selftests/bpf: Add self-tests for new helper bpf_get_ns_current_pid_tgid.") Signed-off-by:
-
Andrii Nakryiko authored
AddressSanitizer assumes that all memory dereferences are done against memory allocated by sanitizer's malloc()/free() code and not touched by anyone else. Seems like this doesn't hold for perf buffer memory. Disable instrumentation on perf buffer callback function. Signed-off-by:
-
Andrii Nakryiko authored
BTF object wasn't freed. Fixes: a6ed02ca ("libbpf: Load btf_vmlinux only once per object.") Signed-off-by:
-
Andrii Nakryiko authored
Another one found by AddressSanitizer. input_len is bigger than actually initialized data size. Fixes: c7566a69 ("selftests/bpf: Add field existence CO-RE relocs tests") Signed-off-by:
-
Andrii Nakryiko authored
getline() allocates string, which has to be freed. Fixes: 81f77fd0 ("bpf: add selftest for stackmap with BPF_F_STACK_BUILD_ID") Signed-off-by:
-
Andrii Nakryiko authored
Free test selector substrings, which were strdup()'ed. Fixes: b65053cd ("selftests/bpf: Add whitelist/blacklist of test names to test_progs") Signed-off-by:
-
Andrii Nakryiko authored
Fix memory leak in hashmap_clear() not freeing hashmap_entry structs for each of the remaining entries. Also NULL-out bucket list to prevent possible double-free between hashmap__clear() and hashmap__free(). Running test_progs-asan flavor clearly showed this problem. Reported-by:
Alston Tang <alston64@fb.com> Signed-off-by:
-
Andrii Nakryiko authored
Fold stand-alone test_hashmap test into test_progs. Signed-off-by:
-
Andrii Nakryiko authored
Add ability to specify extra compiler flags with SAN_CFLAGS for compilation of all user-space C files. This allows to build all of selftest programs with, e.g., custom sanitizer flags, without requiring support for such sanitizers from anyone compiling selftest/bpf. As an example, to compile everything with AddressSanitizer, one would do: $ make clean && make SAN_CFLAGS="-fsanitize=address" For AddressSanitizer to work, one needs appropriate libasan shared library installed in the system, with version of libasan matching what GCC links against. E.g., GCC8 needs libasan5, while GCC7 uses libasan4. For CentOS 7, to build everything successfully one would need to: $ sudo yum install devtoolset-8-gcc devtoolset-libasan-devel $ scl enable devtoolset-8 bash # set up environment For Arch Linux to run selftests, one would need to install gcc-libs package to get libasan.so.5: $ sudo pacman -S gcc-libs N.B. EXTRA_CFLAGS name wasn't used, because it's also used by libbpf's Makefile and this causes few issues: 1. default "-g -Wall" flags are overriden; 2. compiling shared library with AddressSanitizer generates a bunch of symbols like: "_GLOBAL__sub_D_00099_0_btf_dump.c", "_GLOBAL__sub_D_00099_0_bpf.c", etc, which screws up versioned symbols check. Signed-off-by:
-
Andrii Nakryiko authored
Ensure that test runner flavors include their own skeletons from <flavor>/ directory. Previously, skeletons generated for no-flavor test_progs were used. Apart from fixing correctness, this also makes it possible to compile only flavors individually: $ make clean && make test_progs-no_alu32 ... now succeeds ... Fixes: 74b5a596 ("selftests/bpf: Replace test_progs and test_maps w/ general rule") Signed-off-by:
-
Alexei Starovoitov authored
Andrii Nakryiko says: ==================== This patch set teaches libbpf how to declare and initialize ARRAY_OF_MAPS and HASH_OF_MAPS maps. See patch #3 for all the details. Patch #1 refactors parsing BTF definition of map to re-use it cleanly for inner map definition parsing. Patch #2 refactors map creation and destruction logic for reuse. It also fixes existing bug with not closing successfully created maps when bpf_object map creation overall fails. Patch #3 adds support for an extension of BTF-defined map syntax, as well as parsing, recording, and use of relocations to allow declaratively initialize outer maps with references to inner maps. v1->v2: - rename __inner to __array (Alexei). ==================== Signed-off-by: -
Andrii Nakryiko authored
As discussed at LPC 2019 ([0]), this patch brings (a quite belated) support for declarative BTF-defined map-in-map support in libbpf. It allows to define ARRAY_OF_MAPS and HASH_OF_MAPS BPF maps without any user-space initialization code involved. Additionally, it allows to initialize outer map's slots with references to respective inner maps at load time, also completely declaratively. Despite a weak type system of C, the way BTF-defined map-in-map definition works, it's actually quite hard to accidentally initialize outer map with incompatible inner maps. This being C, of course, it's still possible, but even that would be caught at load time and error returned with helpful debug log pointing exactly to the slot that failed to be initialized. As an example, here's a rather advanced HASH_OF_MAPS declaration and initialization example, filling slots #0 and #4 with two inner maps: #include <bpf/bpf_helpers.h> struct inner_map { __uint(type, BPF_MAP_TYPE_ARRAY); __uint(max_entries, 1); __type(key, int); __type(value, int); } inner_map1 SEC(".maps"), inner_map2 SEC(".maps"); struct outer_hash { __uint(type, BPF_MAP_TYPE_HASH_OF_MAPS); __uint(max_entries, 5); __uint(key_size, sizeof(int)); __array(values, struct inner_map); } outer_hash SEC(".maps") = { .values = { [0] = &inner_map2, [4] = &inner_map1, }, }; Here's the relevant part of libbpf debug log showing pretty clearly of what's going on with map-in-map initialization: libbpf: .maps relo #0: for 6 value 0 rel.r_offset 96 name 260 ('inner_map1') libbpf: .maps relo #0: map 'outer_arr' slot [0] points to map 'inner_map1' libbpf: .maps relo #1: for 7 value 32 rel.r_offset 112 name 249 ('inner_map2') libbpf: .maps relo #1: map 'outer_arr' slot [2] points to map 'inner_map2' libbpf: .maps relo #2: for 7 value 32 rel.r_offset 144 name 249 ('inner_map2') libbpf: .maps relo #2: map 'outer_hash' slot [0] points to map 'inner_map2' libbpf: .maps relo #3: for 6 value 0 rel.r_offset 176 name 260 ('inner_map1') libbpf: .maps relo #3: map 'outer_hash' slot [4] points to map 'inner_map1' libbpf: map 'inner_map1': created successfully, fd=4 libbpf: map 'inner_map2': created successfully, fd=5 libbpf: map 'outer_hash': created successfully, fd=7 libbpf: map 'outer_hash': slot [0] set to map 'inner_map2' fd=5 libbpf: map 'outer_hash': slot [4] set to map 'inner_map1' fd=4 Notice from the log above that fd=6 (not logged explicitly) is used for inner "prototype" map, necessary for creation of outer map. It is destroyed immediately after outer map is created. See also included selftest with some extra comments explaining extra details of usage. Additionally, similar initialization syntax and libbpf functionality can be used to do initialization of BPF_PROG_ARRAY with references to BPF sub-programs. This can be done in follow up patches, if there will be a demand for this. [0] https://linuxplumbersconf.org/event/4/contributions/448/Signed-off-by:Toke Høiland-Jørgensen <toke@redhat.com> Link: https://lore.kernel.org/bpf/20200429002739.48006-4-andriin@fb.com
-
Andrii Nakryiko authored
Factor out map creation and destruction logic to simplify code and especially error handling. Also fix map FD leak in case of partially successful map creation during bpf_object load operation. Fixes: 57a00f41 ("libbpf: Add auto-pinning of maps when loading BPF objects") Signed-off-by:
-
Andrii Nakryiko authored
Factor out BTF map definition logic into stand-alone routine for easier reuse for map-in-map case. Signed-off-by:
-
Alexei Starovoitov authored
Andrii Nakryiko says: ==================== This patch series adds various observability APIs to bpf_link: - each bpf_link now gets ID, similar to bpf_map and bpf_prog, by which user-space can iterate over all existing bpf_links and create limited FD from ID; - allows to get extra object information with bpf_link general and type-specific information; - implements `bpf link show` command which lists all active bpf_links in the system; - implements `bpf link pin` allowing to pin bpf_link by ID or from other pinned path. v2->v3: - improve spin locking around bpf_link ID (Alexei); - simplify bpf_link_info handling and fix compilation error on sh arch; v1->v2: - simplified `bpftool link show` implementation (Quentin); - fixed formatting of bpftool-link.rst (Quentin); - fixed attach type printing logic (Quentin); rfc->v1: - dropped read-only bpf_links (Alexei); - fixed bug in bpf_link_cleanup() not removing ID; - fixed bpftool link pinning search logic; - added bash-completion and man page. ==================== Signed-off-by: -
Andrii Nakryiko authored
Extend bpftool's bash-completion script to handle new link command and its sub-commands. Signed-off-by:
-
Andrii Nakryiko authored
Add bpftool-link manpage with information and examples of link-related commands. Signed-off-by:
-
Andrii Nakryiko authored
Add `bpftool link show` and `bpftool link pin` commands. Example plain output for `link show` (with showing pinned paths): [vmuser@archvm bpf]$ sudo ~/local/linux/tools/bpf/bpftool/bpftool -f link 1: tracing prog 12 prog_type tracing attach_type fentry pinned /sys/fs/bpf/my_test_link pinned /sys/fs/bpf/my_test_link2 2: tracing prog 13 prog_type tracing attach_type fentry 3: tracing prog 14 prog_type tracing attach_type fentry 4: tracing prog 15 prog_type tracing attach_type fentry 5: tracing prog 16 prog_type tracing attach_type fentry 6: tracing prog 17 prog_type tracing attach_type fentry 7: raw_tracepoint prog 21 tp 'sys_enter' 8: cgroup prog 25 cgroup_id 584 attach_type egress 9: cgroup prog 25 cgroup_id 599 attach_type egress 10: cgroup prog 25 cgroup_id 614 attach_type egress 11: cgroup prog 25 cgroup_id 629 attach_type egress Signed-off-by: -
Andrii Nakryiko authored
Move attach_type_strings into main.h for access in non-cgroup code. bpf_attach_type is used for non-cgroup attach types quite widely now. So also complete missing string translations for non-cgroup attach types. Signed-off-by:
-
Andrii Nakryiko authored
Extend bpf_obj_id selftest to verify bpf_link's observability APIs. Signed-off-by:
-
Andrii Nakryiko authored
Add low-level API calls for bpf_link_get_next_id() and bpf_link_get_fd_by_id(). Signed-off-by:
-
Andrii Nakryiko authored
Add ability to fetch bpf_link details through BPF_OBJ_GET_INFO_BY_FD command. Also enhance show_fdinfo to potentially include bpf_link type-specific information (similarly to obj_info). Also introduce enum bpf_link_type stored in bpf_link itself and expose it in UAPI. bpf_link_tracing also now will store and return bpf_attach_type. Signed-off-by:
-
