- 09 Feb, 2022 18 commits
-
-
Jakub Sitnicki authored
remote_port is another case of a BPF context field documented as a 32-bit value in network byte order for which the BPF context access converter generates a load of a zero-padded 16-bit integer in network byte order. First such case was dst_port in bpf_sock which got addressed in commit 4421a582 ("bpf: Make dst_port field in struct bpf_sock 16-bit wide"). Loading 4-bytes from the remote_port offset and converting the value with bpf_ntohl() leads to surprising results, as the expected value is shifted by 16 bits. Reduce the confusion by splitting the field in two - a 16-bit field holding a big-endian integer, and a 16-bit zero-padding anonymous field that follows it. Suggested-by:
Alexei Starovoitov <ast@kernel.org> Signed-off-by:
Jakub Sitnicki <jakub@cloudflare.com> Signed-off-by:
-
Andrii Nakryiko authored
On ppc64le architecture __s64 is long int and requires %ld. Cast to ssize_t and use %zd to avoid architecture-specific specifiers. Fixes: 4172843e ("libbpf: Fix signedness bug in btf_dump_array_data()") Signed-off-by:
Andrii Nakryiko <andrii@kernel.org> Signed-off-by:
Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20220209063909.1268319-1-andrii@kernel.org
-
Andrii Nakryiko authored
Hengqi Chen says: ==================== Add new macro BPF_KPROBE_SYSCALL, which provides easy access to syscall input arguments. See [0] and [1] for background. [0]: https://github.com/libbpf/libbpf-bootstrap/issues/57 [1]: https://github.com/libbpf/libbpf/issues/425 v2->v3: - Use PT_REGS_SYSCALL_REGS - Move selftest to progs/bpf_syscall_macro.c v1->v2: - Use PT_REGS_PARM2_CORE_SYSCALL instead ==================== Signed-off-by:
-
Hengqi Chen authored
Add tests for the newly added BPF_KPROBE_SYSCALL macro. Signed-off-by:
Hengqi Chen <hengqi.chen@gmail.com> Signed-off-by:
-
Hengqi Chen authored
Add syscall-specific variant of BPF_KPROBE named BPF_KPROBE_SYSCALL ([0]). The new macro hides the underlying way of getting syscall input arguments. With the new macro, the following code: SEC("kprobe/__x64_sys_close") int BPF_KPROBE(do_sys_close, struct pt_regs *regs) { int fd; fd = PT_REGS_PARM1_CORE(regs); /* do something with fd */ } can be written as: SEC("kprobe/__x64_sys_close") int BPF_KPROBE_SYSCALL(do_sys_close, int fd) { /* do something with fd */ } [0] Closes: https://github.com/libbpf/libbpf/issues/425Signed-off-by: -
Andrii Nakryiko authored
Ilya Leoshkevich says: ==================== libbpf now has macros to access syscall arguments in an architecture-agnostic manner, but unfortunately they have a number of issues on non-Intel arches, which this series aims to fix. v1: https://lore.kernel.org/bpf/20220201234200.1836443-1-iii@linux.ibm.com/ v1 -> v2: * Put orig_gpr2 in place of args[1] on s390 (Vasily). * Fix arm64, powerpc and riscv (Heiko). v2: https://lore.kernel.org/bpf/20220204041955.1958263-1-iii@linux.ibm.com/ v2 -> v3: * Undo args[1] change (Andrii). * Rename PT_REGS_SYSCALL to PT_REGS_SYSCALL_REGS (Andrii). * Split the riscv patch (Andrii). v3: https://lore.kernel.org/bpf/20220204145018.1983773-1-iii@linux.ibm.com/ v3 -> v4: * Undo arm64's and s390's user_pt_regs changes. * Use struct pt_regs when vmlinux.h is available (Andrii). * Use offsetofend for accessing orig_gpr2 and orig_x0 (Andrii). * Move libbpf's copy of offsetofend to a new header. * Fix riscv's __PT_FP_REG. * Use PT_REGS_SYSCALL_REGS in test_probe_user.c. * Test bpf_syscall_macro with userspace headers. * Use Naveen's suggestions and code in patches 5 and 6. * Add warnings to arm64's and s390's ptrace.h (Andrii). v4: https://lore.kernel.org/bpf/20220208051635.2160304-1-iii@linux.ibm.com/ v4 -> v5: * Go back to v3. * Do not touch arch headers. * Use CO-RE struct flavors to access orig_x0 and orig_gpr2. * Fail compilation if non-CO-RE macros are used to access the first syscall parameter on arm64 and s390. * Fix accessing frame pointer on riscv. ==================== Signed-off-by:
-
Ilya Leoshkevich authored
On s390, the first syscall argument should be accessed via orig_gpr2 (see arch/s390/include/asm/syscall.h). Currently gpr[2] is used instead, leading to bpf_syscall_macro test failure. orig_gpr2 cannot be added to user_pt_regs, since its layout is a part of the ABI. Therefore provide access to it only through PT_REGS_PARM1_CORE_SYSCALL() by using a struct pt_regs flavor. Reported-by:
Ilya Leoshkevich <iii@linux.ibm.com> Signed-off-by:
-
Ilya Leoshkevich authored
On arm64, the first syscall argument should be accessed via orig_x0 (see arch/arm64/include/asm/syscall.h). Currently regs[0] is used instead, leading to bpf_syscall_macro test failure. orig_x0 cannot be added to struct user_pt_regs, since its layout is a part of the ABI. Therefore provide access to it only through PT_REGS_PARM1_CORE_SYSCALL() by using a struct pt_regs flavor. Reported-by:
Heiko Carstens <hca@linux.ibm.com> Signed-off-by:
-
Ilya Leoshkevich authored
arm64 and s390 need a special way to access the first syscall argument. Signed-off-by:
-
Ilya Leoshkevich authored
These architectures can provide access to the first syscall argument only through PT_REGS_PARM1_CORE_SYSCALL(). Signed-off-by:
-
Ilya Leoshkevich authored
riscv does not select ARCH_HAS_SYSCALL_WRAPPER, so its syscall handlers take "unpacked" syscall arguments. Indicate this to libbpf using PT_REGS_SYSCALL_REGS macro. Reported-by:
-
Ilya Leoshkevich authored
riscv registers are accessed via struct user_regs_struct, not struct pt_regs. The program counter member in this struct is called pc, not epc. The frame pointer is called s0, not fp. Fixes: 3cc31d79 ("libbpf: Normalize PT_REGS_xxx() macro definitions") Signed-off-by:
-
Ilya Leoshkevich authored
powerpc does not select ARCH_HAS_SYSCALL_WRAPPER, so its syscall handlers take "unpacked" syscall arguments. Indicate this to libbpf using PT_REGS_SYSCALL_REGS macro. Reported-by:
Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com> Link: https://lore.kernel.org/bpf/20220209021745.2215452-5-iii@linux.ibm.com
-
Ilya Leoshkevich authored
Ensure that PT_REGS_SYSCALL_REGS works correctly. Signed-off-by:
-
Ilya Leoshkevich authored
Architectures that select ARCH_HAS_SYSCALL_WRAPPER pass a pointer to struct pt_regs to syscall handlers, others unpack it into individual function parameters. Introduce a macro to describe what a particular arch does. Signed-off-by:
-
Ilya Leoshkevich authored
bpf_syscall_macro reads a long argument into an int variable, which produces a wrong value on big-endian systems. Fix by reading the argument into an intermediate long variable first. Fixes: 77fc0330 ("selftests/bpf: Add a test to confirm PT_REGS_PARM4_SYSCALL") Signed-off-by:
-
Alexei Starovoitov authored
Song Liu says: ==================== Fix build errors reported by kernel test robot. ==================== Signed-off-by: -
Song Liu authored
Fix build with CONFIG_TRANSPARENT_HUGEPAGE=n with BPF_PROG_PACK_SIZE as PAGE_SIZE. Fixes: 57631054 ("bpf: Introduce bpf_prog_pack allocator") Reported-by:
kernel test robot <lkp@intel.com> Signed-off-by:
Song Liu <song@kernel.org> Signed-off-by:
-
- 08 Feb, 2022 16 commits
-
-
Song Liu authored
Replace header->pages * PAGE_SIZE with new header->size. Fixes: ed2d9e1a ("bpf: Use size instead of pages in bpf_binary_header") Reported-by:
-
Dan Carpenter authored
The btf__resolve_size() function returns negative error codes so "elem_size" must be signed for the error handling to work. Fixes: 920d16af ("libbpf: BTF dumper support for typed data") Signed-off-by:
Dan Carpenter <dan.carpenter@oracle.com> Signed-off-by:
Yonghong Song <yhs@fb.com> Link: https://lore.kernel.org/bpf/20220208071552.GB10495@kili
-
Hou Tao authored
Two subtests in ksyms_module.c are not qualified as static, so these subtests are exported as standalone tests in tests.h and lead to confusion for the output of "./test_progs -t ksyms_module". By using the following command ... grep "^void \(serial_\)\?test_[a-zA-Z0-9_]\+(\(void\)\?)" \ tools/testing/selftests/bpf/prog_tests/*.c | \ awk -F : '{print $1}' | sort | uniq -c | awk '$1 != 1' ... one finds out that other tests also have a similar problem, so fix these tests by marking subtests in these tests as static. Signed-off-by:Hou Tao <houtao1@huawei.com> Signed-off-by:
-
Song Liu authored
Instead of BUG_ON(), fail gracefully and return orig_prog. Fixes: 1022a549 ("bpf, x86_64: Use bpf_jit_binary_pack_alloc") Signed-off-by:
-
Stanislav Fomichev authored
This place also uses signed min_t and passes this singed int to copy_to_user (which accepts unsigned argument). I don't think there is an issue, but let's be consistent. Fixes: 7855e0db ("bpf: test_run: add xdp_shared_info pointer in bpf_test_finish signature") Signed-off-by:
Stanislav Fomichev <sdf@google.com> Signed-off-by:
-
Stanislav Fomichev authored
When kattr->test.data_size_in > INT_MAX, signed min_t will assign negative value to data_len. This negative value then gets passed over to copy_from_user where it is converted to (big) unsigned. Use unsigned min_t to avoid this overflow. usercopy: Kernel memory overwrite attempt detected to wrapped address (offset 0, size 18446612140539162846)! ------------[ cut here ]------------ kernel BUG at mm/usercopy.c:102! invalid opcode: 0000 [#1] SMP KASAN Modules linked in: CPU: 0 PID: 3781 Comm: syz-executor226 Not tainted 4.15.0-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 RIP: 0010:usercopy_abort+0xbd/0xbf mm/usercopy.c:102 RSP: 0018:ffff8801e9703a38 EFLAGS: 00010286 RAX: 000000000000006c RBX: ffffffff84fc7040 RCX: 0000000000000000 RDX: 0000000000000000 RSI: ffffffff816560a2 RDI: ffffed003d2e0739 RBP: ffff8801e9703a90 R08: 000000000000006c R09: 0000000000000001 R10: 0000000000000000 R11: 0000000000000000 R12: ffffffff84fc73a0 R13: ffffffff84fc7180 R14: ffffffff84fc7040 R15: ffffffff84fc7040 FS: 00007f54e0bec300(0000) GS:ffff8801f6600000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000020000280 CR3: 00000001e90ea000 CR4: 00000000003426f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: check_bogus_address mm/usercopy.c:155 [inline] __check_object_size mm/usercopy.c:263 [inline] __check_object_size.cold+0x8c/0xad mm/usercopy.c:253 check_object_size include/linux/thread_info.h:112 [inline] check_copy_size include/linux/thread_info.h:143 [inline] copy_from_user include/linux/uaccess.h:142 [inline] bpf_prog_test_run_xdp+0xe57/0x1240 net/bpf/test_run.c:989 bpf_prog_test_run kernel/bpf/syscall.c:3377 [inline] __sys_bpf+0xdf2/0x4a50 kernel/bpf/syscall.c:4679 SYSC_bpf kernel/bpf/syscall.c:4765 [inline] SyS_bpf+0x26/0x50 kernel/bpf/syscall.c:4763 do_syscall_64+0x21a/0x3e0 arch/x86/entry/common.c:305 entry_SYSCALL_64_after_hwframe+0x46/0xbb Fixes: 1c194998 ("bpf: introduce frags support to bpf_prog_test_run_xdp()") Reported-by:
syzbot <syzkaller@googlegroups.com> Signed-off-by:
Lorenzo Bianconi <lorenzo@kernel.org> Acked-by:
-
Alexei Starovoitov authored
Song Liu says: ==================== Changes v8 => v9: 1. Fix an error with multi function program, in 4/9. Changes v7 => v8: 1. Rebase and fix conflicts. 2. Lock text_mutex for text_poke_copy. (Daniel) Changes v6 => v7: 1. Redesign the interface between generic and arch logic, based on feedback from Alexei and Ilya. 2. Split 6/7 of v6 to 7/9 and 8/9 in v7, for cleaner logic. 3. Add bpf_arch_text_copy in 6/9. Changes v5 => v6: 1. Make jit_hole_buffer 128 byte long. Only fill the first and last 128 bytes of header with INT3. (Alexei) 2. Use kvmalloc for temporary buffer. (Alexei) 3. Rename tmp_header/tmp_image => rw_header/rw_image. Remove tmp_image from x64_jit_data. (Alexei) 4. Change fall back round_up_to in bpf_jit_binary_alloc_pack() from BPF_PROG_MAX_PACK_PROG_SIZE to PAGE_SIZE. Changes v4 => v5: 1. Do not use atomic64 for bpf_jit_current. (Alexei) Changes v3 => v4: 1. Rename text_poke_jit() => text_poke_copy(). (Peter) 2. Change comment style. (Peter) Changes v2 => v3: 1. Fix tailcall. Changes v1 => v2: 1. Use text_poke instead of writing through linear mapping. (Peter) 2. Avoid making changes to non-x86_64 code. Most BPF programs are small, but they consume a page each. For systems with busy traffic and many BPF programs, this could also add significant pressure to instruction TLB. High iTLB pressure usually causes slow down for the whole system, which includes visible performance degradation for production workloads. This set tries to solve this problem with customized allocator that pack multiple programs into a huge page. Patches 1-6 prepare the work. Patch 7 contains key logic of bpf_prog_pack allocator. Patch 8 contains bpf_jit_binary_pack_alloc logic on top of bpf_prog_pack allocator. Patch 9 uses this allocator in x86_64 jit. ==================== Signed-off-by: -
Song Liu authored
Use bpf_jit_binary_pack_alloc in x86_64 jit. The jit engine first writes the program to the rw buffer. When the jit is done, the program is copied to the final location with bpf_jit_binary_pack_finalize. Note that we need to do bpf_tail_call_direct_fixup after finalize. Therefore, the text_live = false logic in __bpf_arch_text_poke is no longer needed. Signed-off-by:
Song Liu <songliubraving@fb.com> Signed-off-by:
-
Song Liu authored
This is the jit binary allocator built on top of bpf_prog_pack. bpf_prog_pack allocates RO memory, which cannot be used directly by the JIT engine. Therefore, a temporary rw buffer is allocated for the JIT engine. Once JIT is done, bpf_jit_binary_pack_finalize is used to copy the program to the RO memory. bpf_jit_binary_pack_alloc reserves 16 bytes of extra space for illegal instructions, which is small than the 128 bytes space reserved by bpf_jit_binary_alloc. This change is necessary for bpf_jit_binary_hdr to find the correct header. Also, flag use_bpf_prog_pack is added to differentiate a program allocated by bpf_jit_binary_pack_alloc. Signed-off-by:
-
Song Liu authored
Most BPF programs are small, but they consume a page each. For systems with busy traffic and many BPF programs, this could add significant pressure to instruction TLB. High iTLB pressure usually causes slow down for the whole system, which includes visible performance degradation for production workloads. Introduce bpf_prog_pack allocator to pack multiple BPF programs in a huge page. The memory is then allocated in 64 byte chunks. Memory allocated by bpf_prog_pack allocator is RO protected after initial allocation. To write to it, the user (jit engine) need to use text poke API. Signed-off-by:
-
Song Liu authored
This will be used to copy JITed text to RO protected module memory. On x86, bpf_arch_text_copy is implemented with text_poke_copy. bpf_arch_text_copy returns pointer to dst on success, and ERR_PTR(errno) on errors. Signed-off-by:
-
Song Liu authored
This will be used by BPF jit compiler to dump JITed binary to a RX huge page, and thus allow multiple BPF programs sharing the a huge (2MB) page. Signed-off-by:
Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lore.kernel.org/bpf/20220204185742.271030-6-song@kernel.org
-
Song Liu authored
Using prog->jited_len is simpler and more accurate than current estimation (header + header->size). Also, fix missing prog->jited_len with multi function program. This hasn't been a real issue before this. Signed-off-by:
-
Song Liu authored
This is necessary to charge sub page memory for the BPF program. Signed-off-by:
-
Song Liu authored
This enables sub-page memory charge and allocation. Signed-off-by:
-
Song Liu authored
This enables module_alloc() to allocate huge page for 2MB+ requests. To check the difference of this change, we need enable config CONFIG_PTDUMP_DEBUGFS, and call module_alloc(2MB). Before the change, /sys/kernel/debug/page_tables/kernel shows pte for this map. With the change, /sys/kernel/debug/page_tables/ show pmd for thie map. Signed-off-by:
-
- 07 Feb, 2022 6 commits
-
-
Andrii Nakryiko authored
Mauricio Vásquez <mauricio@kinvolk.io> says: ==================== This series fixes a bad calculation of strict mode in two places. It also updates libbpf to make it easier for the users to disable a specific LIBBPF_STRICT_* flag. v1 -> v2: - remove check in libbpf_set_strict_mode() - split in different commits v1: https://lore.kernel.org/bpf/20220204220435.301896-1-mauricio@kinvolk.io/ ==================== Signed-off-by:
-
Mauricio Vásquez authored
"(__LIBBPF_STRICT_LAST - 1) & ~LIBBPF_STRICT_MAP_DEFINITIONS" is wrong as it is equal to 0 (LIBBPF_STRICT_NONE). Let's use "LIBBPF_STRICT_ALL & ~LIBBPF_STRICT_MAP_DEFINITIONS" now that the previous commit makes it possible in libbpf. Fixes: 93b8952d ("libbpf: deprecate legacy BPF map definitions") Signed-off-by:
Mauricio Vásquez <mauricio@kinvolk.io> Signed-off-by:
-
Mauricio Vásquez authored
"(__LIBBPF_STRICT_LAST - 1) & ~LIBBPF_STRICT_MAP_DEFINITIONS" is wrong as it is equal to 0 (LIBBPF_STRICT_NONE). Let's use "LIBBPF_STRICT_ALL & ~LIBBPF_STRICT_MAP_DEFINITIONS" now that the previous commit makes it possible in libbpf. Fixes: 93b8952d ("libbpf: deprecate legacy BPF map definitions") Signed-off-by:
-
Mauricio Vásquez authored
libbpf_set_strict_mode() checks that the passed mode doesn't contain extra bits for LIBBPF_STRICT_* flags that don't exist yet. It makes it difficult for applications to disable some strict flags as something like "LIBBPF_STRICT_ALL & ~LIBBPF_STRICT_MAP_DEFINITIONS" is rejected by this check and they have to use a rather complicated formula to calculate it.[0] One possibility is to change LIBBPF_STRICT_ALL to only contain the bits of all existing LIBBPF_STRICT_* flags instead of 0xffffffff. However it's not possible because the idea is that applications compiled against older libbpf_legacy.h would still be opting into latest LIBBPF_STRICT_ALL features.[1] The other possibility is to remove that check so something like "LIBBPF_STRICT_ALL & ~LIBBPF_STRICT_MAP_DEFINITIONS" is allowed. It's what this commit does. [0]: https://lore.kernel.org/bpf/20220204220435.301896-1-mauricio@kinvolk.io/ [1]: https://lore.kernel.org/bpf/CAEf4BzaTWa9fELJLh+bxnOb0P1EMQmaRbJVG0L+nXZdy0b8G3Q@mail.gmail.com/ Fixes: 93b8952d ("libbpf: deprecate legacy BPF map definitions") Signed-off-by:
-
Naveen N. Rao authored
Some of the tests are using x86_64 ABI-specific syscall entry points (such as __x64_sys_nanosleep and __x64_sys_getpgid). Update them to use architecture-dependent syscall entry names. Also update fexit_sleep test to not use BPF_PROG() so that it is clear that the syscall parameters aren't being accessed in the bpf prog. Note that none of the bpf progs in these tests are actually accessing any of the syscall parameters. The only exception is perfbuf_bench, which passes on the bpf prog context into bpf_perf_event_output() as a pointer to pt_regs, but that looks to be mostly ignored. Signed-off-by:
-
Naveen N. Rao authored
On architectures that don't use a syscall wrapper, sys_* function names are set as an alias of __se_sys_* functions. Due to this, there is no BTF associated with sys_* function names. This results in some of the test progs failing to load. Set the SYS_PREFIX to "__se_" to fix this issue. Fixes: 38261f36 ("selftests/bpf: Fix probe_user test failure with clang build kernel") Signed-off-by:
-
