- 23 May, 2018 2 commits
-
-
Martin KaFai Lau authored
There are currently unused section descriptions in the btf_header. Those sections are here to support future BTF use cases. For example, the func section (func_off) is to support function signature (e.g. the BPF prog function signature). Instead of spelling out all potential sections up-front in the btf_header. This patch makes changes to btf_header such that extending it (e.g. adding a section) is possible later. The unused ones can be removed for now and they can be added back later. This patch: 1. adds a hdr_len to the btf_header. It will allow adding sections (and other info like parent_label and parent_name) later. The check is similar to the existing bpf_attr. If a user passes in a longer hdr_len, the kernel ensures the extra tailing bytes are 0. 2. allows the section order in the BTF object to be different from its sec_off order in btf_header. 3. each sec_off is followed by a sec_len. It must not have gap or overlapping among sections. The string section is ensured to be at the end due to the 4 bytes alignment requirement of the type section. The above changes will allow enough flexibility to add new sections (and other info) to the btf_header later. This patch also removes an unnecessary !err check at the end of btf_parse(). Signed-off-by:
Martin KaFai Lau <kafai@fb.com> Signed-off-by:
Daniel Borkmann <daniel@iogearbox.net>
-
Martin KaFai Lau authored
This patch exposes check_uarg_tail_zero() which will be reused by a later BTF patch. Its name is changed to bpf_check_uarg_tail_zero(). Signed-off-by:
Yonghong Song <yhs@fb.com> Signed-off-by:
-
- 22 May, 2018 13 commits
-
-
Daniel Borkmann authored
David Ahern says: ==================== Packets that exceed the egress MTU can not be forwarded in the fast path. Add IPv4 and IPv6 MTU helpers that take a FIB lookup result (versus the typical dst path) and add the calls to bpf_ipv{4,6}_fib_lookup. v2 - add ip6_mtu_from_fib6 to ipv6_stub - only call the new MTU helpers for fib lookups in XDP path; skb path uses is_skb_forwardable to determine if the packet can be sent via the egress device from the FIB lookup ==================== Signed-off-by: -
David Ahern authored
Add check that egress MTU can handle packet to be forwarded. If the MTU is less than the packet length, return 0 meaning the packet is expected to continue up the stack for help - eg., fragmenting the packet or sending an ICMP. The XDP path needs to leverage the FIB entry for an MTU on the route spec or an exception entry for a given destination. The skb path lets is_skb_forwardable decide if the packet can be sent. Signed-off-by:
David Ahern <dsahern@gmail.com> Signed-off-by:
-
David Ahern authored
Determine path MTU from a FIB lookup result. Logic is based on ip6_dst_mtu_forward plus lookup of nexthop exception. Add ip6_dst_mtu_forward to ipv6_stubs to handle access by core bpf code. Signed-off-by:
-
David Ahern authored
Determine path MTU from a FIB lookup result. Logic is a distillation of ip_dst_mtu_maybe_forward. Signed-off-by:
-
Daniel Borkmann authored
Björn Töpel says: ==================== This the second follow-up set. The first four patches are uapi changes: * Removing rebind support * Getting rid of structure hole * Removing explicit cache line alignment * Stricter bind checks The last patches do some cleanups, where the umem and refcount_t changes were suggested by Daniel. * Add a missing write-barrier and use READ_ONCE for data-dependencies * Clean up umem and do proper locking * Convert atomic_t to refcount_t ==================== Signed-off-by: -
Björn Töpel authored
Introduce refcount_t, in favor of atomic_t. Signed-off-by:
Björn Töpel <bjorn.topel@intel.com> Signed-off-by:
-
Björn Töpel authored
As suggested by Daniel Borkmann, the umem setup code was a too defensive and complex. Here, we reduce the number of checks. Also, the memory pinning is now folded into the umem creation, and we do correct locking. Signed-off-by:
-
Björn Töpel authored
Here, we add a missing write-barrier, and use READ_ONCE for the data-dependency barrier. Signed-off-by:
-
Björn Töpel authored
Adapt xdpsock to use the new getsockopt introduced in the previous commit. Signed-off-by:
-
Björn Töpel authored
In this commit we remove the explicit ring structure from the the uapi. It is tricky for an uapi to depend on a certain L1 cache line size, since it can differ for variants of the same architecture. Now, we let the user application determine the offsets of the producer, consumer and descriptors by asking the socket via getsockopt. A typical flow would be (Rx ring): struct xdp_mmap_offsets off; struct xdp_desc *ring; u32 *prod, *cons; void *map; ... getsockopt(fd, SOL_XDP, XDP_MMAP_OFFSETS, &off, &optlen); map = mmap(NULL, off.rx.desc + NUM_DESCS * sizeof(struct xdp_desc), PROT_READ | PROT_WRITE, MAP_SHARED | MAP_POPULATE, sfd, XDP_PGOFF_RX_RING); prod = map + off.rx.producer; cons = map + off.rx.consumer; ring = map + off.rx.desc; Signed-off-by:
-
Magnus Karlsson authored
Validate the queue id against both Rx and Tx on the netdev. Also, make sure that the queue exists at xmit time. Reported-by:
Jesper Dangaard Brouer <brouer@redhat.com> Tested-by:
Magnus Karlsson <magnus.karlsson@intel.com> Signed-off-by:
-
Björn Töpel authored
Move the sxdp_flags up, avoiding a hole in the uapi structure. Signed-off-by:
-
Björn Töpel authored
Supporting rebind, i.e. after a successful bind the process can call bind again without closing the socket, makes the AF_XDP setup state machine more complex. Constrain the state space, by not supporting rebind. Signed-off-by:
-
- 18 May, 2018 12 commits
-
-
Daniel Borkmann authored
John Fastabend says: ==================== In this series we add the ability for sk msg programs to read basic sock information about the sock they are attached to. The second patch adds the tests to the selftest test_verifier. One observation that I had from writing this seriess is lots of the ./net/core/filter.c code is almost duplicated across program types. I thought about building a template/macro that we could use as a single block of code to read sock data out for multiple programs, but I wasn't convinced it was worth it yet. The result was using a macro saved a couple lines of code per block but made the code a bit harder to read IMO. We can probably revisit the idea later if we get more duplication. v2: add errstr field to negative test_verifier test cases to ensure we get the expected err string back from the verifier. ==================== Signed-off-by: -
John Fastabend authored
Add tests for BPF_PROG_TYPE_SK_MSG to test_verifier for read access to new sk fields. Signed-off-by:
John Fastabend <john.fastabend@gmail.com> Acked-by:
-
John Fastabend authored
Currently sk_msg programs only have access to the raw data. However, it is often useful when building policies to have the policies specific to the socket endpoint. This allows using the socket tuple as input into filters, etc. This patch adds ctx access to the sock fields. Signed-off-by:
-
Daniel Borkmann authored
Jiong Wang says: ==================== NFP eBPF JIT is missing logic indirect shifts (both left and right) and arithmetic right shift (both indirect shift and shift by constant). This patch adds support for them. For indirect shifts, shift amount is not specified as constant, NFP needs to get the shift amount through the low 5 bits of source A operand in PREV_ALU, therefore extra instructions are needed compared with shifts by constants. Because NFP is 32-bit, so we are using register pair for 64-bit shifts and therefore would need different instruction sequences depending on whether shift amount is less than 32 or not. NFP branch-on-bit-test instruction emitter is added by this patch set and is used for efficient runtime check on shift amount. We'd think the shift amount is less than 32 if bit 5 is clear and greater or equal then 32 otherwise. Shift amount is greater than or equal to 64 will result in undefined behavior. This patch also use range info to avoid generating unnecessary runtime code if we are certain shift amount is less than 32 or not. ==================== Signed-off-by: -
Jiong Wang authored
Code logic is similar with arithmetic right shift by constant, and NFP get indirect shift amount through source A operand of PREV_ALU. It is possible to fall back to logic right shift if the MSB is known to be zero from range info, however there is no benefit to do this given logic indirect right shift use the same number and cycle of instruction sequence. Suppose the MSB of regX is the bit we want to replicate to fill in all the vacant positions, and regY contains the shift amount, then we could use single instruction to set up both. [alu, --, regY, OR, regX] -- NOTE: the PREV_ALU result doesn't need to write to any destination register. Signed-off-by:Jiong Wang <jiong.wang@netronome.com> Reviewed-by:
Jakub Kicinski <jakub.kicinski@netronome.com> Signed-off-by:
-
Jiong Wang authored
Code logic is similar with logic right shift except we also need to set PREV_ALU result properly, the MSB of which is the bit that will be replicated to fill in all the vacant positions. Signed-off-by:
-
Jiong Wang authored
For indirect shifts, shift amount is not specified as constant, NFP needs to get the shift amount through the low 5 bits of source A operand in PREV_ALU, therefore extra instructions are needed compared with shifts by constants. Because NFP is 32-bit, so we are using register pair for 64-bit shifts and therefore would need different instruction sequences depending on whether shift amount is less than 32 or not. NFP branch-on-bit-test instruction emitter is added by this patch and is used for efficient runtime check on shift amount. We'd think the shift amount is less than 32 if bit 5 is clear and greater or equal than 32 otherwise. Shift amount is greater than or equal to 64 will result in undefined behavior. This patch also use range info to avoid generating unnecessary runtime code if we are certain shift amount is less than 32 or not. Signed-off-by:
-
Daniel Borkmann authored
Björn Töpel says: ==================== This series contain "cosmetics only" follow-up patches for AF_XDP. Thanks to Daniel for suggesting them! ==================== Signed-off-by: -
Björn Töpel authored
Properly align xsk_proto_ops initialization. Signed-off-by:
-
Björn Töpel authored
Removed some cases of unnecessary parentheses. Signed-off-by:
-
Björn Töpel authored
Minor cleanup, remove newline at end of Makefile. Signed-off-by:
-
Björn Töpel authored
Clean up SPDX-License-Identifier and removing licensing leftovers. Signed-off-by:
-
- 17 May, 2018 4 commits
-
-
Gustavo A. R. Silva authored
`e' is being freed twice. Fix this by removing one of the kfree() calls. Addresses-Coverity-ID: 1468983 ("Double free") Fixes: 81110384 ("bpf: sockmap, add hash map support") Signed-off-by:Gustavo A. R. Silva <gustavo@embeddedor.com> Acked-by:
-
Gustavo A. R. Silva authored
There is a potential execution path in which variable err is returned without being properly initialized previously. Fix this by initializing variable err to 0. Addresses-Coverity-ID: 1468964 ("Uninitialized scalar variable") Fixes: e5cd3abc ("bpf: sockmap, refactor sockmap routines to work with hashmap") Signed-off-by: -
Quentin Monnet authored
Documentation for eBPF helpers can be parsed from bpf.h and eventually turned into a man page. Commit 6f96674d ("bpf: relax constraints on formatting for eBPF helper documentation") changed the script used to parse it, in order to allow for different indent style and to ease the work for writing documentation for future helpers. The script currently considers that the first tab can be replaced by 6 to 8 spaces. But the documentation for bpf_fib_lookup() uses a mix of tabs (for the "Description" part) and of spaces ("Return" part), and only has 5 space long indent for the latter. We probably do not want to change the values accepted by the script each time a new helper gets a new indent style. However, it is worth noting that with those 5 spaces, the "Description" and "Return" part *look* aligned in the generated patch and in `git show`, so it is likely other helper authors will use the same length. Therefore, allow for helper documentation to use 5 spaces only for the first indent level. Signed-off-by:
Quentin Monnet <quentin.monnet@netronome.com> Signed-off-by:
-
git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-nextDavid S. Miller authored
Daniel Borkmann says: ==================== pull-request: bpf-next 2018-05-17 The following pull-request contains BPF updates for your *net-next* tree. The main changes are: 1) Provide a new BPF helper for doing a FIB and neighbor lookup in the kernel tables from an XDP or tc BPF program. The helper provides a fast-path for forwarding packets. The API supports IPv4, IPv6 and MPLS protocols, but currently IPv4 and IPv6 are implemented in this initial work, from David (Ahern). 2) Just a tiny diff but huge feature enabled for nfp driver by extending the BPF offload beyond a pure host processing offload. Offloaded XDP programs are allowed to set the RX queue index and thus opening the door for defining a fully programmable RSS/n-tuple filter replacement. Once BPF decided on a queue already, the device data-path will skip the conventional RSS processing completely, from Jakub. 3) The original sockmap implementation was array based similar to devmap. However unlike devmap where an ifindex has a 1:1 mapping into the map there are use cases with sockets that need to be referenced using longer keys. Hence, sockhash map is added reusing as much of the sockmap code as possible, from John. 4) Introduce BTF ID. The ID is allocatd through an IDR similar as with BPF maps and progs. It also makes BTF accessible to user space via BPF_BTF_GET_FD_BY_ID and adds exposure of the BTF data through BPF_OBJ_GET_INFO_BY_FD, from Martin. 5) Enable BPF stackmap with build_id also in NMI context. Due to the up_read() of current->mm->mmap_sem build_id cannot be parsed. This work defers the up_read() via a per-cpu irq_work so that at least limited support can be enabled, from Song. 6) Various BPF JIT follow-up cleanups and fixups after the LD_ABS/LD_IND JIT conversion as well as implementation of an optimized 32/64 bit immediate load in the arm64 JIT that allows to reduce the number of emitted instructions; in case of tested real-world programs they were shrinking by three percent, from Daniel. 7) Add ifindex parameter to the libbpf loader in order to enable BPF offload support. Right now only iproute2 can load offloaded BPF and this will also enable libbpf for direct integration into other applications, from David (Beckett). 8) Convert the plain text documentation under Documentation/bpf/ into RST format since this is the appropriate standard the kernel is moving to for all documentation. Also add an overview README.rst, from Jesper. 9) Add __printf verification attribute to the bpf_verifier_vlog() helper. Though it uses va_list we can still allow gcc to check the format string, from Mathieu. 10) Fix a bash reference in the BPF selftest's Makefile. The '|& ...' is a bash 4.0+ feature which is not guaranteed to be available when calling out to shell, therefore use a more portable variant, from Joe. 11) Fix a 64 bit division in xdp_umem_reg() by using div_u64() instead of relying on the gcc built-in, from Björn. 12) Fix a sock hashmap kmalloc warning reported by syzbot when an overly large key size is used in hashmap then causing overflows in htab->elem_size. Reject bogus attr->key_size early in the sock_hash_alloc(), from Yonghong. 13) Ensure in BPF selftests when urandom_read is being linked that --build-id is always enabled so that test_stacktrace_build_id[_nmi] won't be failing, from Alexei. 14) Add bitsperlong.h as well as errno.h uapi headers into the tools header infrastructure which point to one of the arch specific uapi headers. This was needed in order to fix a build error on some systems for the BPF selftests, from Sirio. 15) Allow for short options to be used in the xdp_monitor BPF sample code. And also a bpf.h tools uapi header sync in order to fix a selftest build failure. Both from Prashant. 16) More formally clarify the meaning of ID in the direct packet access section of the BPF documentation, from Wang. ==================== Signed-off-by:David S. Miller <davem@davemloft.net>
-
- 16 May, 2018 9 commits
-
-
John Fastabend authored
When an error happens in the update sockmap element logic also pass the err up to the user. Fixes: e5cd3abc ("bpf: sockmap, refactor sockmap routines to work with hashmap") Signed-off-by:
-
Yonghong Song authored
syzbot reported a kernel warning below: WARNING: CPU: 0 PID: 4499 at mm/slab_common.c:996 kmalloc_slab+0x56/0x70 mm/slab_common.c:996 Kernel panic - not syncing: panic_on_warn set ... CPU: 0 PID: 4499 Comm: syz-executor050 Not tainted 4.17.0-rc3+ #9 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0x1b9/0x294 lib/dump_stack.c:113 panic+0x22f/0x4de kernel/panic.c:184 __warn.cold.8+0x163/0x1b3 kernel/panic.c:536 report_bug+0x252/0x2d0 lib/bug.c:186 fixup_bug arch/x86/kernel/traps.c:178 [inline] do_error_trap+0x1de/0x490 arch/x86/kernel/traps.c:296 do_invalid_op+0x1b/0x20 arch/x86/kernel/traps.c:315 invalid_op+0x14/0x20 arch/x86/entry/entry_64.S:992 RIP: 0010:kmalloc_slab+0x56/0x70 mm/slab_common.c:996 RSP: 0018:ffff8801d907fc58 EFLAGS: 00010246 RAX: 0000000000000000 RBX: ffff8801aeecb280 RCX: ffffffff8185ebd7 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 00000000ffffffe1 RBP: ffff8801d907fc58 R08: ffff8801adb5e1c0 R09: ffffed0035a84700 R10: ffffed0035a84700 R11: ffff8801ad423803 R12: ffff8801aeecb280 R13: 00000000fffffff4 R14: ffff8801ad891a00 R15: 00000000014200c0 __do_kmalloc mm/slab.c:3713 [inline] __kmalloc+0x25/0x760 mm/slab.c:3727 kmalloc include/linux/slab.h:517 [inline] map_get_next_key+0x24a/0x640 kernel/bpf/syscall.c:858 __do_sys_bpf kernel/bpf/syscall.c:2131 [inline] __se_sys_bpf kernel/bpf/syscall.c:2096 [inline] __x64_sys_bpf+0x354/0x4f0 kernel/bpf/syscall.c:2096 do_syscall_64+0x1b1/0x800 arch/x86/entry/common.c:287 entry_SYSCALL_64_after_hwframe+0x49/0xbe The test case is against sock hashmap with a key size 0xffffffe1. Such a large key size will cause the below code in function sock_hash_alloc() overflowing and produces a smaller elem_size, hence map creation will be successful. htab->elem_size = sizeof(struct htab_elem) + round_up(htab->map.key_size, 8); Later, when map_get_next_key is called and kernel tries to allocate the key unsuccessfully, it will issue the above warning. Similar to hashtab, ensure the key size is at most MAX_BPF_STACK for a successful map creation. Fixes: 81110384 ("bpf: sockmap, add hash map support") Reported-by: syzbot+e4566d29080e7f3460ff@syzkaller.appspotmail.com Signed-off-by: -
David Beckett authored
BPF programs currently can only be offloaded using iproute2. This patch will allow programs to be offloaded using libbpf calls. Signed-off-by:
David Beckett <david.beckett@netronome.com> Reviewed-by:
-
Mathieu Malaterre authored
__printf is useful to verify format and arguments. ‘bpf_verifier_vlog’ function is used twice in verifier.c in both cases the caller function already uses the __printf gcc attribute. Remove the following warning, triggered with W=1: kernel/bpf/verifier.c:176:2: warning: function might be possible candidate for ‘gnu_printf’ format attribute [-Wsuggest-attribute=format] Signed-off-by:
Mathieu Malaterre <malat@debian.org> Signed-off-by:
-
David Ahern authored
Only consider forwarding packets if ttl in received packet is > 1 and decrement ttl before handing off to bpf_redirect_map. Signed-off-by:
-
Daniel Borkmann authored
John Fastabend says: ==================== In the original sockmap implementation we got away with using an array similar to devmap. However, unlike devmap where an ifindex has a nice 1:1 function into the map we have found some use cases with sockets that need to be referenced using longer keys. This series adds support for a sockhash map reusing as much of the sockmap code as possible. I made the decision to add sockhash specific helpers vs trying to generalize the existing helpers because (a) they have sockmap in the name and (b) the keys are different types. I prefer to be explicit here rather than play type games or do something else tricky. To test this we duplicate all the sockmap testing except swap out the sockmap with a sockhash. v2: fix file stats and add v2 tag v3: move tool updates into test patch, move bpftool updates into its own patch, and fixup the test patch stats to catch the renamed file and provide only diffs ± on that. v4: Add documentation to UAPI bpf.h v5: Add documentation to tools UAPI bpf.h v6: 'git add' test_sockhash_kern.c which was previously missing but was not causing issues because of typo in test script, noticed by Daniel. After this the git format-patch -M option no longer tracks the rename of the test_sockmap_kern files for some reason. I guess the diff has exceeded some threshold. ==================== Signed-off-by: -
John Fastabend authored
This adds the SOCKHASH map type to bpftools so that we get correct pretty printing. Signed-off-by:
-
John Fastabend authored
This runs existing SOCKMAP tests with SOCKHASH map type. To do this we push programs into include file and build two BPF programs. One for SOCKHASH and one for SOCKMAP. We then run the entire test suite with each type. Signed-off-by:
-
Rahul Lakkireddy authored
For T6, clip table is separated from main TCAM. So, update LE-TCAM collection logic to collect clip table TCAM as well. IPv6 takes 4 entries in clip table TCAM compared to 2 entries in main TCAM. Also, in case of errors, keep LE-TCAM collected so far and set the status to partial dump. Signed-off-by:
Rahul Lakkireddy <rahul.lakkireddy@chelsio.com> Signed-off-by:
Ganesh Goudar <ganeshgr@chelsio.com> Signed-off-by:
-
