Skip to content

L
linux
Commit ab0720ce authored by Andrii Nakryiko's avatar Andrii Nakryiko
Browse files
Options

Merge branch 'tools: bpftool: update, synchronise and validate types and options' 

Quentin Monnet says:

====================

To work with the different program types, map types, attach types etc.
supported by eBPF, bpftool needs occasional updates to learn about the new
features supported by the kernel. When such types translate into new
keyword for the command line, updates are expected in several locations:
typically, the help message displayed from bpftool itself, the manual page,
and the bash completion file should be updated. The options used by the
different commands for bpftool should also remain synchronised at those
locations.

Several omissions have occurred in the past, and a number of types are
still missing today. This set is an attempt to improve the situation. It
brings up-to-date the lists of types or options in bpftool, and also adds a
Python script to the BPF selftests to automatically check that most of
these lists remain synchronised.

v2:
- Reformat some lines in the bash completion file.
- Do not reformat attach types, to preserve git-blame history.
- Do not call Python script from tools/testing/selftests/bpf/Makefile.
====================
Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
parents 0b846445 475a23c2
Hide whitespace changes
Inline Side-by-side
Showing with 763 additions and 61 deletions
tools/bpf/bpftool/Documentation/bpftool-btf.rst
View file @ ab0720ce
......@@ -12,7 +12,8 @@ SYNOPSIS
**bpftool** [*OPTIONS*] **btf** *COMMAND*
*OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] }
*OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] | {**-d** | **--debug** } |
{ **-B** | **--base-btf** } }
*COMMANDS* := { **dump** | **help** }
......@@ -73,6 +74,20 @@ OPTIONS
=======
.. include:: common_options.rst
-B, --base-btf *FILE*
Pass a base BTF object. Base BTF objects are typically used
with BTF objects for kernel modules. To avoid duplicating
all kernel symbols required by modules, BTF objects for
modules are "split", they are built incrementally on top of
the kernel (vmlinux) BTF object. So the base BTF reference
should usually point to the kernel BTF.
When the main BTF object to process (for example, the
module BTF to dump) is passed as a *FILE*, bpftool attempts
to autodetect the path for the base object, and passing
this option is optional. When the main BTF object is passed
through other handles, this option becomes necessary.
EXAMPLES
========
**# bpftool btf dump id 1226**
......@@ -217,3 +232,34 @@ All the standard ways to specify map or program are supported:
**# bpftool btf dump prog tag b88e0a09b1d9759d**
**# bpftool btf dump prog pinned /sys/fs/bpf/prog_name**
|
| **# bpftool btf dump file /sys/kernel/btf/i2c_smbus**
| (or)
| **# I2C_SMBUS_ID=$(bpftool btf show -p | jq '.[] | select(.name=="i2c_smbus").id')**
| **# bpftool btf dump id ${I2C_SMBUS_ID} -B /sys/kernel/btf/vmlinux**
::
[104848] STRUCT 'i2c_smbus_alert' size=40 vlen=2
'alert' type_id=393 bits_offset=0
'ara' type_id=56050 bits_offset=256
[104849] STRUCT 'alert_data' size=12 vlen=3
'addr' type_id=16 bits_offset=0
'type' type_id=56053 bits_offset=32
'data' type_id=7 bits_offset=64
[104850] PTR '(anon)' type_id=104848
[104851] PTR '(anon)' type_id=104849
[104852] FUNC 'i2c_register_spd' type_id=84745 linkage=static
[104853] FUNC 'smbalert_driver_init' type_id=1213 linkage=static
[104854] FUNC_PROTO '(anon)' ret_type_id=18 vlen=1
'ara' type_id=56050
[104855] FUNC 'i2c_handle_smbus_alert' type_id=104854 linkage=static
[104856] FUNC 'smbalert_remove' type_id=104854 linkage=static
[104857] FUNC_PROTO '(anon)' ret_type_id=18 vlen=2
'ara' type_id=56050
'id' type_id=56056
[104858] FUNC 'smbalert_probe' type_id=104857 linkage=static
[104859] FUNC 'smbalert_work' type_id=9695 linkage=static
[104860] FUNC 'smbus_alert' type_id=71367 linkage=static
[104861] FUNC 'smbus_do_alert' type_id=84827 linkage=static
tools/bpf/bpftool/Documentation/bpftool-cgroup.rst
View file @ ab0720ce
......@@ -12,7 +12,8 @@ SYNOPSIS
**bpftool** [*OPTIONS*] **cgroup** *COMMAND*
*OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] | { **-f** | **--bpffs** } }
*OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] | { **-d** | **--debug** } |
{ **-f** | **--bpffs** } }
*COMMANDS* :=
{ **show** | **list** | **tree** | **attach** | **detach** | **help** }
......
tools/bpf/bpftool/Documentation/bpftool-feature.rst
View file @ ab0720ce
......@@ -12,7 +12,7 @@ SYNOPSIS
**bpftool** [*OPTIONS*] **feature** *COMMAND*
*OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] }
*OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] | { **-d** | **--debug** } }
*COMMANDS* := { **probe** | **help** }
......
tools/bpf/bpftool/Documentation/bpftool-gen.rst
View file @ ab0720ce
......@@ -12,7 +12,8 @@ SYNOPSIS
**bpftool** [*OPTIONS*] **gen** *COMMAND*
*OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] }
*OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] | { **-d** | **--debug** } |
{ **-L** | **--use-loader** } }
*COMMAND* := { **object** | **skeleton** | **help** }
......@@ -152,6 +153,12 @@ OPTIONS
=======
.. include:: common_options.rst
-L, --use-loader
For skeletons, generate a "light" skeleton (also known as "loader"
skeleton). A light skeleton contains a loader eBPF program. It does
not use the majority of the libbpf infrastructure, and does not need
libelf.
EXAMPLES
========
**$ cat example1.bpf.c**
......
tools/bpf/bpftool/Documentation/bpftool-iter.rst
View file @ ab0720ce
......@@ -12,6 +12,8 @@ SYNOPSIS
**bpftool** [*OPTIONS*] **iter** *COMMAND*
*OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] | { **-d** | **--debug** } }
*COMMANDS* := { **pin** | **help** }
ITER COMMANDS
......
tools/bpf/bpftool/Documentation/bpftool-link.rst
View file @ ab0720ce
......@@ -12,7 +12,8 @@ SYNOPSIS
**bpftool** [*OPTIONS*] **link** *COMMAND*
*OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] | { **-f** | **--bpffs** } }
*OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] | { **-d** | **--debug** } |
{ **-f** | **--bpffs** } | { **-n** | **--nomount** } }
*COMMANDS* := { **show** | **list** | **pin** | **help** }
......
tools/bpf/bpftool/Documentation/bpftool-map.rst
View file @ ab0720ce
......@@ -12,7 +12,8 @@ SYNOPSIS
**bpftool** [*OPTIONS*] **map** *COMMAND*
*OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] | { **-f** | **--bpffs** } }
*OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] | { **-d** | **--debug** } |
{ **-f** | **--bpffs** } | { **-n** | **--nomount** } }
*COMMANDS* :=
{ **show** | **list** | **create** | **dump** | **update** | **lookup** | **getnext**
......
tools/bpf/bpftool/Documentation/bpftool-net.rst
View file @ ab0720ce
......@@ -12,7 +12,7 @@ SYNOPSIS
**bpftool** [*OPTIONS*] **net** *COMMAND*
*OPTIONS* := { [{ **-j** | **--json** }] [{ **-p** | **--pretty** }] }
*OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] | { **-d** | **--debug** } }
*COMMANDS* :=
{ **show** | **list** | **attach** | **detach** | **help** }
......
tools/bpf/bpftool/Documentation/bpftool-perf.rst
View file @ ab0720ce
......@@ -12,7 +12,7 @@ SYNOPSIS
**bpftool** [*OPTIONS*] **perf** *COMMAND*
*OPTIONS* := { [{ **-j** | **--json** }] [{ **-p** | **--pretty** }] }
*OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] | { **-d** | **--debug** } }
*COMMANDS* :=
{ **show** | **list** | **help** }
......
tools/bpf/bpftool/Documentation/bpftool-prog.rst
View file @ ab0720ce
......@@ -12,7 +12,9 @@ SYNOPSIS
**bpftool** [*OPTIONS*] **prog** *COMMAND*
*OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] | { **-f** | **--bpffs** } }
*OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] | { **-d** | **--debug** } |
{ **-f** | **--bpffs** } | { **-m** | **--mapcompat** } | { **-n** | **--nomount** } |
{ **-L** | **--use-loader** } }
*COMMANDS* :=
{ **show** | **list** | **dump xlated** | **dump jited** | **pin** | **load**
......@@ -48,10 +50,11 @@ PROG COMMANDS
| **struct_ops** | **fentry** | **fexit** | **freplace** | **sk_lookup**
| }
| *ATTACH_TYPE* := {
| **msg_verdict** | **stream_verdict** | **stream_parser** | **flow_dissector**
| **msg_verdict** | **skb_verdict** | **stream_verdict** | **stream_parser** | **flow_dissector**
| }
| *METRICs* := {
| **cycles** | **instructions** | **l1d_loads** | **llc_misses**
| **cycles** | **instructions** | **l1d_loads** | **llc_misses** |
| **itlb_misses** | **dtlb_misses**
| }
......@@ -223,6 +226,20 @@ OPTIONS
Do not automatically attempt to mount any virtual file system
(such as tracefs or BPF virtual file system) when necessary.
-L, --use-loader
Load program as a "loader" program. This is useful to debug
the generation of such programs. When this option is in
use, bpftool attempts to load the programs from the object
file into the kernel, but does not pin them (therefore, the
*PATH* must not be provided).
When combined with the **-d**\ \|\ **--debug** option,
additional debug messages are generated, and the execution
of the loader program will use the **bpf_trace_printk**\ ()
helper to log each step of loading BTF, creating the maps,
and loading the programs (see **bpftool prog tracelog** as
a way to dump those messages).
EXAMPLES
========
**# bpftool prog show**
......@@ -326,3 +343,16 @@ EXAMPLES
40176203 cycles (83.05%)
42518139 instructions # 1.06 insns per cycle (83.39%)
123 llc_misses # 2.89 LLC misses per million insns (83.15%)
|
| Output below is for the trace logs.
| Run in separate terminals:
| **# bpftool prog tracelog**
| **# bpftool prog load -L -d file.o**
::
bpftool-620059 [004] d... 2634685.517903: bpf_trace_printk: btf_load size 665 r=5
bpftool-620059 [004] d... 2634685.517912: bpf_trace_printk: map_create sample_map idx 0 type 2 value_size 4 value_btf_id 0 r=6
bpftool-620059 [004] d... 2634685.517997: bpf_trace_printk: prog_load sample insn_cnt 13 r=7
bpftool-620059 [004] d... 2634685.517999: bpf_trace_printk: close(5) = 0
tools/bpf/bpftool/Documentation/bpftool-struct_ops.rst
View file @ ab0720ce
......@@ -12,7 +12,7 @@ SYNOPSIS
**bpftool** [*OPTIONS*] **struct_ops** *COMMAND*
*OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] }
*OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] | { **-d** | **--debug** } }
*COMMANDS* :=
{ **show** | **list** | **dump** | **register** | **unregister** | **help** }
......
tools/bpf/bpftool/Documentation/bpftool.rst
View file @ ab0720ce
......@@ -18,15 +18,15 @@ SYNOPSIS
*OBJECT* := { **map** | **program** | **cgroup** | **perf** | **net** | **feature** }
*OPTIONS* := { { **-V** | **--version** } | { **-h** | **--help** }
| { **-j** | **--json** } [{ **-p** | **--pretty** }] }
*OPTIONS* := { { **-V** | **--version** } |
{ **-j** | **--json** } [{ **-p** | **--pretty** }] | { **-d** | **--debug** } }
*MAP-COMMANDS* :=
{ **show** | **list** | **create** | **dump** | **update** | **lookup** | **getnext**
| **delete** | **pin** | **event_pipe** | **help** }
{ **show** | **list** | **create** | **dump** | **update** | **lookup** | **getnext** |
**delete** | **pin** | **event_pipe** | **help** }
*PROG-COMMANDS* := { **show** | **list** | **dump jited** | **dump xlated** | **pin**
| **load** | **attach** | **detach** | **help** }
*PROG-COMMANDS* := { **show** | **list** | **dump jited** | **dump xlated** | **pin** |
**load** | **attach** | **detach** | **help** }
*CGROUP-COMMANDS* := { **show** | **list** | **attach** | **detach** | **help** }
......
tools/bpf/bpftool/bash-completion/bpftool
View file @ ab0720ce
......@@ -260,7 +260,8 @@ _bpftool()
# Deal with options
if [[ ${words[cword]} == -* ]]; then
local c='--version --json --pretty --bpffs --mapcompat --debug'
local c='--version --json --pretty --bpffs --mapcompat --debug \
--use-loader --base-btf'
COMPREPLY=( $( compgen -W "$c" -- "$cur" ) )
return 0
fi
......@@ -278,7 +279,7 @@ _bpftool()
_sysfs_get_netdevs
return 0
;;
file|pinned)
file|pinned|-B|--base-btf)
_filedir
return 0
;;
......@@ -291,7 +292,8 @@ _bpftool()
# Remove all options so completions don't have to deal with them.
local i
for (( i=1; i < ${#words[@]}; )); do
if [[ ${words[i]::1} == - ]]; then
if [[ ${words[i]::1} == - ]] &&
[[ ${words[i]} != "-B" ]] && [[ ${words[i]} != "--base-btf" ]]; then
words=( "${words[@]:0:i}" "${words[@]:i+1}" )
[[ $i -le $cword ]] && cword=$(( cword - 1 ))
else
......@@ -343,7 +345,8 @@ _bpftool()
local PROG_TYPE='id pinned tag name'
local MAP_TYPE='id pinned name'
local METRIC_TYPE='cycles instructions l1d_loads llc_misses'
local METRIC_TYPE='cycles instructions l1d_loads llc_misses \
itlb_misses dtlb_misses'
case $command in
show|list)
[[ $prev != "$command" ]] && return 0
......@@ -404,8 +407,10 @@ _bpftool()
return 0
;;
5)
COMPREPLY=( $( compgen -W 'msg_verdict stream_verdict \
stream_parser flow_dissector' -- "$cur" ) )
local BPFTOOL_PROG_ATTACH_TYPES='msg_verdict \
skb_verdict stream_verdict stream_parser \
flow_dissector'
COMPREPLY=( $( compgen -W "$BPFTOOL_PROG_ATTACH_TYPES" -- "$cur" ) )
return 0
;;
6)
......@@ -464,7 +469,7 @@ _bpftool()
case $prev in
type)
COMPREPLY=( $( compgen -W "socket kprobe \
local BPFTOOL_PROG_LOAD_TYPES='socket kprobe \
kretprobe classifier flow_dissector \
action tracepoint raw_tracepoint \
xdp perf_event cgroup/skb cgroup/sock \
......@@ -479,8 +484,8 @@ _bpftool()
cgroup/post_bind4 cgroup/post_bind6 \
cgroup/sysctl cgroup/getsockopt \
cgroup/setsockopt cgroup/sock_release struct_ops \
fentry fexit freplace sk_lookup" -- \
"$cur" ) )
fentry fexit freplace sk_lookup'
COMPREPLY=( $( compgen -W "$BPFTOOL_PROG_LOAD_TYPES" -- "$cur" ) )
return 0
;;
id)
......@@ -698,15 +703,15 @@ _bpftool()
return 0
;;
type)
COMPREPLY=( $( compgen -W 'hash array prog_array \
perf_event_array percpu_hash percpu_array \
stack_trace cgroup_array lru_hash \
local BPFTOOL_MAP_CREATE_TYPES='hash array \
prog_array perf_event_array percpu_hash \
percpu_array stack_trace cgroup_array lru_hash \
lru_percpu_hash lpm_trie array_of_maps \
hash_of_maps devmap devmap_hash sockmap cpumap \
xskmap sockhash cgroup_storage reuseport_sockarray \
percpu_cgroup_storage queue stack sk_storage \
struct_ops inode_storage task_storage' -- \
"$cur" ) )
struct_ops inode_storage task_storage ringbuf'
COMPREPLY=( $( compgen -W "$BPFTOOL_MAP_CREATE_TYPES" -- "$cur" ) )
return 0
;;
key|value|flags|entries)
......@@ -1017,34 +1022,37 @@ _bpftool()
return 0
;;
attach|detach)
local ATTACH_TYPES='ingress egress sock_create sock_ops \
device bind4 bind6 post_bind4 post_bind6 connect4 connect6 \
local BPFTOOL_CGROUP_ATTACH_TYPES='ingress egress \
sock_create sock_ops device \
bind4 bind6 post_bind4 post_bind6 connect4 connect6 \
getpeername4 getpeername6 getsockname4 getsockname6 \
sendmsg4 sendmsg6 recvmsg4 recvmsg6 sysctl getsockopt \
setsockopt sock_release'
local ATTACH_FLAGS='multi override'
local PROG_TYPE='id pinned tag name'
case $prev in
$command)
_filedir
return 0
;;
ingress|egress|sock_create|sock_ops|device|bind4|bind6|\
post_bind4|post_bind6|connect4|connect6|getpeername4|\
getpeername6|getsockname4|getsockname6|sendmsg4|sendmsg6|\
recvmsg4|recvmsg6|sysctl|getsockopt|setsockopt|sock_release)
# Check for $prev = $command first
if [ $prev = $command ]; then
_filedir
return 0
# Then check for attach type. This is done outside of the
# "case $prev in" to avoid writing the whole list of attach
# types again as pattern to match (where we cannot reuse
# our variable).
elif [[ $BPFTOOL_CGROUP_ATTACH_TYPES =~ $prev ]]; then
COMPREPLY=( $( compgen -W "$PROG_TYPE" -- \
"$cur" ) )
return 0
;;
fi
# case/esac for the other cases
case $prev in
id)
_bpftool_get_prog_ids
return 0
;;
*)
if ! _bpftool_search_list "$ATTACH_TYPES"; then
COMPREPLY=( $( compgen -W "$ATTACH_TYPES" -- \
"$cur" ) )
if ! _bpftool_search_list "$BPFTOOL_CGROUP_ATTACH_TYPES"; then
COMPREPLY=( $( compgen -W \
"$BPFTOOL_CGROUP_ATTACH_TYPES" -- "$cur" ) )
elif [[ "$command" == "attach" ]]; then
# We have an attach type on the command line,
# but it is not the previous word, or
......
tools/bpf/bpftool/btf.c
View file @ ab0720ce
......@@ -981,7 +981,8 @@ static int do_help(int argc, char **argv)
" FORMAT := { raw | c }\n"
" " HELP_SPEC_MAP "\n"
" " HELP_SPEC_PROGRAM "\n"
" " HELP_SPEC_OPTIONS "\n"
" " HELP_SPEC_OPTIONS " |\n"
" {-B|--base-btf} }\n"
"",
bin_name, "btf");
......
tools/bpf/bpftool/cgroup.c
View file @ ab0720ce
......@@ -501,7 +501,8 @@ static int do_help(int argc, char **argv)
HELP_SPEC_ATTACH_TYPES "\n"
" " HELP_SPEC_ATTACH_FLAGS "\n"
" " HELP_SPEC_PROGRAM "\n"
" " HELP_SPEC_OPTIONS "\n"
" " HELP_SPEC_OPTIONS " |\n"
" {-f|--bpffs} }\n"
"",
bin_name, argv[-2]);
......
tools/bpf/bpftool/common.c
View file @ ab0720ce
......@@ -67,6 +67,12 @@ const char * const attach_type_name[__MAX_BPF_ATTACH_TYPE] = {
[BPF_MODIFY_RETURN] = "mod_ret",
[BPF_LSM_MAC] = "lsm_mac",
[BPF_SK_LOOKUP] = "sk_lookup",
[BPF_TRACE_ITER] = "trace_iter",
[BPF_XDP_DEVMAP] = "xdp_devmap",
[BPF_XDP_CPUMAP] = "xdp_cpumap",
[BPF_XDP] = "xdp",
[BPF_SK_REUSEPORT_SELECT] = "sk_skb_reuseport_select",
[BPF_SK_REUSEPORT_SELECT_OR_MIGRATE] = "sk_skb_reuseport_select_or_migrate",
};
void p_err(const char *fmt, ...)
......
tools/bpf/bpftool/feature.c
View file @ ab0720ce
......@@ -1005,6 +1005,7 @@ static int do_help(int argc, char **argv)
" %1$s %2$s help\n"
"\n"
" COMPONENT := { kernel | dev NAME }\n"
" " HELP_SPEC_OPTIONS " }\n"
"",
bin_name, argv[-2]);
......
tools/bpf/bpftool/gen.c
View file @ ab0720ce
......@@ -1026,7 +1026,8 @@ static int do_help(int argc, char **argv)
" %1$s %2$s skeleton FILE [name OBJECT_NAME]\n"
" %1$s %2$s help\n"
"\n"
" " HELP_SPEC_OPTIONS "\n"
" " HELP_SPEC_OPTIONS " |\n"
" {-L|--use-loader} }\n"
"",
bin_name, "gen");
......
tools/bpf/bpftool/iter.c
View file @ ab0720ce
......@@ -97,7 +97,9 @@ static int do_help(int argc, char **argv)
fprintf(stderr,
"Usage: %1$s %2$s pin OBJ PATH [map MAP]\n"
" %1$s %2$s help\n"
"\n"
" " HELP_SPEC_MAP "\n"
" " HELP_SPEC_OPTIONS " }\n"
"",
bin_name, "iter");
......
tools/bpf/bpftool/link.c
View file @ ab0720ce
......@@ -401,7 +401,8 @@ static int do_help(int argc, char **argv)
" %1$s %2$s help\n"
"\n"
" " HELP_SPEC_LINK "\n"
" " HELP_SPEC_OPTIONS "\n"
" " HELP_SPEC_OPTIONS " |\n"
" {-f|--bpffs} | {-n|--nomount} }\n"
"",
bin_name, argv[-2]);
......
tools/bpf/bpftool/main.c
View file @ ab0720ce
......@@ -64,7 +64,8 @@ static int do_help(int argc, char **argv)
" %s version\n"
"\n"
" OBJECT := { prog | map | link | cgroup | perf | net | feature | btf | gen | struct_ops | iter }\n"
" " HELP_SPEC_OPTIONS "\n"
" " HELP_SPEC_OPTIONS " |\n"
" {-V|--version} }\n"
"",
bin_name, bin_name, bin_name);
......
tools/bpf/bpftool/main.h
View file @ ab0720ce
......@@ -57,8 +57,7 @@ static inline void *u64_to_ptr(__u64 ptr)
#define HELP_SPEC_PROGRAM \
"PROG := { id PROG_ID | pinned FILE | tag PROG_TAG | name PROG_NAME }"
#define HELP_SPEC_OPTIONS \
"OPTIONS := { {-j|--json} [{-p|--pretty}] | {-f|--bpffs} |\n" \
"\t {-m|--mapcompat} | {-n|--nomount} }"
"OPTIONS := { {-j|--json} [{-p|--pretty}] | {-d|--debug}"
#define HELP_SPEC_MAP \
"MAP := { id MAP_ID | pinned FILE | name MAP_NAME }"
#define HELP_SPEC_LINK \
......
tools/bpf/bpftool/map.c
View file @ ab0720ce
......@@ -1466,8 +1466,9 @@ static int do_help(int argc, char **argv)
" devmap | devmap_hash | sockmap | cpumap | xskmap | sockhash |\n"
" cgroup_storage | reuseport_sockarray | percpu_cgroup_storage |\n"
" queue | stack | sk_storage | struct_ops | ringbuf | inode_storage |\n"
" task_storage }\n"
" " HELP_SPEC_OPTIONS "\n"
" task_storage }\n"
" " HELP_SPEC_OPTIONS " |\n"
" {-f|--bpffs} | {-n|--nomount} }\n"
"",
bin_name, argv[-2]);
......
tools/bpf/bpftool/net.c
View file @ ab0720ce
......@@ -729,6 +729,7 @@ static int do_help(int argc, char **argv)
"\n"
" " HELP_SPEC_PROGRAM "\n"
" ATTACH_TYPE := { xdp | xdpgeneric | xdpdrv | xdpoffload }\n"
" " HELP_SPEC_OPTIONS " }\n"
"\n"
"Note: Only xdp and tc attachments are supported now.\n"
" For progs attached to cgroups, use \"bpftool cgroup\"\n"
......
tools/bpf/bpftool/perf.c
View file @ ab0720ce
......@@ -231,7 +231,10 @@ static int do_show(int argc, char **argv)
static int do_help(int argc, char **argv)
{
fprintf(stderr,
"Usage: %1$s %2$s { show | list | help }\n"
"Usage: %1$s %2$s { show | list }\n"
" %1$s %2$s help }\n"
"\n"
" " HELP_SPEC_OPTIONS " }\n"
"",
bin_name, argv[-2]);
......
tools/bpf/bpftool/prog.c
View file @ ab0720ce
......@@ -2256,10 +2256,12 @@ static int do_help(int argc, char **argv)
" cgroup/sendmsg6 | cgroup/recvmsg4 | cgroup/recvmsg6 |\n"
" cgroup/getsockopt | cgroup/setsockopt | cgroup/sock_release |\n"
" struct_ops | fentry | fexit | freplace | sk_lookup }\n"
" ATTACH_TYPE := { msg_verdict | stream_verdict | stream_parser |\n"
" flow_dissector }\n"
" ATTACH_TYPE := { msg_verdict | skb_verdict | stream_verdict |\n"
" stream_parser | flow_dissector }\n"
" METRIC := { cycles | instructions | l1d_loads | llc_misses | itlb_misses | dtlb_misses }\n"
" " HELP_SPEC_OPTIONS "\n"
" " HELP_SPEC_OPTIONS " |\n"
" {-f|--bpffs} | {-m|--mapcompat} | {-n|--nomount} |\n"
" {-L|--use-loader} }\n"
"",
bin_name, argv[-2]);
......
tools/bpf/bpftool/struct_ops.c
View file @ ab0720ce
......@@ -572,8 +572,8 @@ static int do_help(int argc, char **argv)
" %1$s %2$s unregister STRUCT_OPS_MAP\n"
" %1$s %2$s help\n"
"\n"
" OPTIONS := { {-j|--json} [{-p|--pretty}] }\n"
" STRUCT_OPS_MAP := [ id STRUCT_OPS_MAP_ID | name STRUCT_OPS_MAP_NAME ]\n"
" " HELP_SPEC_OPTIONS " }\n"
"",
bin_name, argv[-2]);
......
tools/testing/selftests/bpf/test_bpftool_synctypes.py 0 → 100755
View file @ ab0720ce
#!/usr/bin/env python3
# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
#
# Copyright (C) 2021 Isovalent, Inc.
import argparse
import re
import os, sys
LINUX_ROOT = os.path.abspath(os.path.join(__file__,
os.pardir, os.pardir, os.pardir, os.pardir, os.pardir))
BPFTOOL_DIR = os.path.join(LINUX_ROOT, 'tools/bpf/bpftool')
retval = 0
class BlockParser(object):
"""
A parser for extracting set of values from blocks such as enums.
@reader: a pointer to the open file to parse
"""
def __init__(self, reader):
self.reader = reader
def search_block(self, start_marker):
"""
Search for a given structure in a file.
@start_marker: regex marking the beginning of a structure to parse
"""
offset = self.reader.tell()
array_start = re.search(start_marker, self.reader.read())
if array_start is None:
raise Exception('Failed to find start of block')
self.reader.seek(offset + array_start.start())
def parse(self, pattern, end_marker):
"""
Parse a block and return a set of values. Values to extract must be
on separate lines in the file.
@pattern: pattern used to identify the values to extract
@end_marker: regex marking the end of the block to parse
"""
entries = set()
while True:
line = self.reader.readline()
if not line or re.match(end_marker, line):
break
capture = pattern.search(line)
if capture and pattern.groups >= 1:
entries.add(capture.group(1))
return entries
class ArrayParser(BlockParser):
"""
A parser for extracting dicionaries of values from some BPF-related arrays.
@reader: a pointer to the open file to parse
@array_name: name of the array to parse
"""
end_marker = re.compile('^};')
def __init__(self, reader, array_name):
self.array_name = array_name
self.start_marker = re.compile(f'(static )?const char \* const {self.array_name}\[.*\] = {{\n')
super().__init__(reader)
def search_block(self):
"""
Search for the given array in a file.
"""
super().search_block(self.start_marker);
def parse(self):
"""
Parse a block and return data as a dictionary. Items to extract must be
on separate lines in the file.
"""
pattern = re.compile('\[(BPF_\w*)\]\s*= "(.*)",?$')
entries = {}
while True:
line = self.reader.readline()
if line == '' or re.match(self.end_marker, line):
break
capture = pattern.search(line)
if capture:
entries[capture.group(1)] = capture.group(2)
return entries
class InlineListParser(BlockParser):
"""
A parser for extracting set of values from inline lists.
"""
def parse(self, pattern, end_marker):
"""
Parse a block and return a set of values. Multiple values to extract
can be on a same line in the file.
@pattern: pattern used to identify the values to extract
@end_marker: regex marking the end of the block to parse
"""
entries = set()
while True:
line = self.reader.readline()
if not line:
break
entries.update(pattern.findall(line))
if re.search(end_marker, line):
break
return entries
class FileExtractor(object):
"""
A generic reader for extracting data from a given file. This class contains
several helper methods that wrap arround parser objects to extract values
from different structures.
This class does not offer a way to set a filename, which is expected to be
defined in children classes.
"""
def __init__(self):
self.reader = open(self.filename, 'r')
def close(self):
"""
Close the file used by the parser.
"""
self.reader.close()
def reset_read(self):
"""
Reset the file position indicator for this parser. This is useful when
parsing several structures in the file without respecting the order in
which those structures appear in the file.
"""
self.reader.seek(0)
def get_types_from_array(self, array_name):
"""
Search for and parse an array associating names to BPF_* enum members,
for example:
const char * const prog_type_name[] = {
[BPF_PROG_TYPE_UNSPEC] = "unspec",
[BPF_PROG_TYPE_SOCKET_FILTER] = "socket_filter",
[BPF_PROG_TYPE_KPROBE] = "kprobe",
};
Return a dictionary with the enum member names as keys and the
associated names as values, for example:
{'BPF_PROG_TYPE_UNSPEC': 'unspec',
'BPF_PROG_TYPE_SOCKET_FILTER': 'socket_filter',
'BPF_PROG_TYPE_KPROBE': 'kprobe'}
@array_name: name of the array to parse
"""
array_parser = ArrayParser(self.reader, array_name)
array_parser.search_block()
return array_parser.parse()
def get_enum(self, enum_name):
"""
Search for and parse an enum containing BPF_* members, for example:
enum bpf_prog_type {
BPF_PROG_TYPE_UNSPEC,
BPF_PROG_TYPE_SOCKET_FILTER,
BPF_PROG_TYPE_KPROBE,
};
Return a set containing all member names, for example:
{'BPF_PROG_TYPE_UNSPEC',
'BPF_PROG_TYPE_SOCKET_FILTER',
'BPF_PROG_TYPE_KPROBE'}
@enum_name: name of the enum to parse
"""
start_marker = re.compile(f'enum {enum_name} {{\n')
pattern = re.compile('^\s*(BPF_\w+),?$')
end_marker = re.compile('^};')
parser = BlockParser(self.reader)
parser.search_block(start_marker)
return parser.parse(pattern, end_marker)
def __get_description_list(self, start_marker, pattern, end_marker):
parser = InlineListParser(self.reader)
parser.search_block(start_marker)
return parser.parse(pattern, end_marker)
def get_rst_list(self, block_name):
"""
Search for and parse a list of type names from RST documentation, for
example:
| *TYPE* := {
| **socket** | **kprobe** |
| **kretprobe**
| }
Return a set containing all type names, for example:
{'socket', 'kprobe', 'kretprobe'}
@block_name: name of the blog to parse, 'TYPE' in the example
"""
start_marker = re.compile(f'\*{block_name}\* := {{')
pattern = re.compile('\*\*([\w/-]+)\*\*')
end_marker = re.compile('}\n')
return self.__get_description_list(start_marker, pattern, end_marker)
def get_help_list(self, block_name):
"""
Search for and parse a list of type names from a help message in
bpftool, for example:
" TYPE := { socket | kprobe |\\n"
" kretprobe }\\n"
Return a set containing all type names, for example:
{'socket', 'kprobe', 'kretprobe'}
@block_name: name of the blog to parse, 'TYPE' in the example
"""
start_marker = re.compile(f'"\s*{block_name} := {{')
pattern = re.compile('([\w/]+) [|}]')
end_marker = re.compile('}')
return self.__get_description_list(start_marker, pattern, end_marker)
def get_help_list_macro(self, macro):
"""
Search for and parse a list of values from a help message starting with
a macro in bpftool, for example:
" " HELP_SPEC_OPTIONS " |\\n"
" {-f|--bpffs} | {-m|--mapcompat} | {-n|--nomount} }\\n"
Return a set containing all item names, for example:
{'-f', '--bpffs', '-m', '--mapcompat', '-n', '--nomount'}
@macro: macro starting the block, 'HELP_SPEC_OPTIONS' in the example
"""
start_marker = re.compile(f'"\s*{macro}\s*" [|}}]')
pattern = re.compile('([\w-]+) ?(?:\||}[ }\]])')
end_marker = re.compile('}\\\\n')
return self.__get_description_list(start_marker, pattern, end_marker)
def default_options(self):
"""
Return the default options contained in HELP_SPEC_OPTIONS
"""
return { '-j', '--json', '-p', '--pretty', '-d', '--debug' }
def get_bashcomp_list(self, block_name):
"""
Search for and parse a list of type names from a variable in bash
completion file, for example:
local BPFTOOL_PROG_LOAD_TYPES='socket kprobe \\
kretprobe'
Return a set containing all type names, for example:
{'socket', 'kprobe', 'kretprobe'}
@block_name: name of the blog to parse, 'TYPE' in the example
"""
start_marker = re.compile(f'local {block_name}=\'')
pattern = re.compile('(?:.*=\')?([\w/]+)')
end_marker = re.compile('\'$')
return self.__get_description_list(start_marker, pattern, end_marker)
class SourceFileExtractor(FileExtractor):
"""
An abstract extractor for a source file with usage message.
This class does not offer a way to set a filename, which is expected to be
defined in children classes.
"""
def get_options(self):
return self.default_options().union(self.get_help_list_macro('HELP_SPEC_OPTIONS'))
class ProgFileExtractor(SourceFileExtractor):
"""
An extractor for bpftool's prog.c.
"""
filename = os.path.join(BPFTOOL_DIR, 'prog.c')
def get_prog_types(self):
return self.get_types_from_array('prog_type_name')
def get_attach_types(self):
return self.get_types_from_array('attach_type_strings')
def get_prog_attach_help(self):
return self.get_help_list('ATTACH_TYPE')
class MapFileExtractor(SourceFileExtractor):
"""
An extractor for bpftool's map.c.
"""
filename = os.path.join(BPFTOOL_DIR, 'map.c')
def get_map_types(self):
return self.get_types_from_array('map_type_name')
def get_map_help(self):
return self.get_help_list('TYPE')
class CgroupFileExtractor(SourceFileExtractor):
"""
An extractor for bpftool's cgroup.c.
"""
filename = os.path.join(BPFTOOL_DIR, 'cgroup.c')
def get_prog_attach_help(self):
return self.get_help_list('ATTACH_TYPE')
class CommonFileExtractor(SourceFileExtractor):
"""
An extractor for bpftool's common.c.
"""
filename = os.path.join(BPFTOOL_DIR, 'common.c')
def __init__(self):
super().__init__()
self.attach_types = {}
def get_attach_types(self):
if not self.attach_types:
self.attach_types = self.get_types_from_array('attach_type_name')
return self.attach_types
def get_cgroup_attach_types(self):
if not self.attach_types:
self.get_attach_types()
cgroup_types = {}
for (key, value) in self.attach_types.items():
if key.find('BPF_CGROUP') != -1:
cgroup_types[key] = value
return cgroup_types
class GenericSourceExtractor(SourceFileExtractor):
"""
An extractor for generic source code files.
"""
filename = ""
def __init__(self, filename):
self.filename = os.path.join(BPFTOOL_DIR, filename)
super().__init__()
class BpfHeaderExtractor(FileExtractor):
"""
An extractor for the UAPI BPF header.
"""
filename = os.path.join(LINUX_ROOT, 'tools/include/uapi/linux/bpf.h')
def get_prog_types(self):
return self.get_enum('bpf_prog_type')
def get_map_types(self):
return self.get_enum('bpf_map_type')
def get_attach_types(self):
return self.get_enum('bpf_attach_type')
class ManPageExtractor(FileExtractor):
"""
An abstract extractor for an RST documentation page.
This class does not offer a way to set a filename, which is expected to be
defined in children classes.
"""
def get_options(self):
return self.get_rst_list('OPTIONS')
class ManProgExtractor(ManPageExtractor):
"""
An extractor for bpftool-prog.rst.
"""
filename = os.path.join(BPFTOOL_DIR, 'Documentation/bpftool-prog.rst')
def get_attach_types(self):
return self.get_rst_list('ATTACH_TYPE')
class ManMapExtractor(ManPageExtractor):
"""
An extractor for bpftool-map.rst.
"""
filename = os.path.join(BPFTOOL_DIR, 'Documentation/bpftool-map.rst')
def get_map_types(self):
return self.get_rst_list('TYPE')
class ManCgroupExtractor(ManPageExtractor):
"""
An extractor for bpftool-cgroup.rst.
"""
filename = os.path.join(BPFTOOL_DIR, 'Documentation/bpftool-cgroup.rst')
def get_attach_types(self):
return self.get_rst_list('ATTACH_TYPE')
class ManGenericExtractor(ManPageExtractor):
"""
An extractor for generic RST documentation pages.
"""
filename = ""
def __init__(self, filename):
self.filename = os.path.join(BPFTOOL_DIR, filename)
super().__init__()
class BashcompExtractor(FileExtractor):
"""
An extractor for bpftool's bash completion file.
"""
filename = os.path.join(BPFTOOL_DIR, 'bash-completion/bpftool')
def get_prog_attach_types(self):
return self.get_bashcomp_list('BPFTOOL_PROG_ATTACH_TYPES')
def get_map_types(self):
return self.get_bashcomp_list('BPFTOOL_MAP_CREATE_TYPES')
def get_cgroup_attach_types(self):
return self.get_bashcomp_list('BPFTOOL_CGROUP_ATTACH_TYPES')
def verify(first_set, second_set, message):
"""
Print all values that differ between two sets.
@first_set: one set to compare
@second_set: another set to compare
@message: message to print for values belonging to only one of the sets
"""
global retval
diff = first_set.symmetric_difference(second_set)
if diff:
print(message, diff)
retval = 1
def main():
# No arguments supported at this time, but print usage for -h|--help
argParser = argparse.ArgumentParser(description="""
Verify that bpftool's code, help messages, documentation and bash
completion are all in sync on program types, map types, attach types, and
options. Also check that bpftool is in sync with the UAPI BPF header.
""")
args = argParser.parse_args()
# Map types (enum)
bpf_info = BpfHeaderExtractor()
ref = bpf_info.get_map_types()
map_info = MapFileExtractor()
source_map_items = map_info.get_map_types()
map_types_enum = set(source_map_items.keys())
verify(ref, map_types_enum,
f'Comparing BPF header (enum bpf_map_type) and {MapFileExtractor.filename} (map_type_name):')
# Map types (names)
source_map_types = set(source_map_items.values())
source_map_types.discard('unspec')
help_map_types = map_info.get_map_help()
help_map_options = map_info.get_options()
map_info.close()
man_map_info = ManMapExtractor()
man_map_options = man_map_info.get_options()
man_map_types = man_map_info.get_map_types()
man_map_info.close()
bashcomp_info = BashcompExtractor()
bashcomp_map_types = bashcomp_info.get_map_types()
verify(source_map_types, help_map_types,
f'Comparing {MapFileExtractor.filename} (map_type_name) and {MapFileExtractor.filename} (do_help() TYPE):')
verify(source_map_types, man_map_types,
f'Comparing {MapFileExtractor.filename} (map_type_name) and {ManMapExtractor.filename} (TYPE):')
verify(help_map_options, man_map_options,
f'Comparing {MapFileExtractor.filename} (do_help() OPTIONS) and {ManMapExtractor.filename} (OPTIONS):')
verify(source_map_types, bashcomp_map_types,
f'Comparing {MapFileExtractor.filename} (map_type_name) and {BashcompExtractor.filename} (BPFTOOL_MAP_CREATE_TYPES):')
# Program types (enum)
ref = bpf_info.get_prog_types()
prog_info = ProgFileExtractor()
prog_types = set(prog_info.get_prog_types().keys())
verify(ref, prog_types,
f'Comparing BPF header (enum bpf_prog_type) and {ProgFileExtractor.filename} (prog_type_name):')
# Attach types (enum)
ref = bpf_info.get_attach_types()
bpf_info.close()
common_info = CommonFileExtractor()
attach_types = common_info.get_attach_types()
verify(ref, attach_types,
f'Comparing BPF header (enum bpf_attach_type) and {CommonFileExtractor.filename} (attach_type_name):')
# Attach types (names)
source_prog_attach_types = set(prog_info.get_attach_types().values())
help_prog_attach_types = prog_info.get_prog_attach_help()
help_prog_options = prog_info.get_options()
prog_info.close()
man_prog_info = ManProgExtractor()
man_prog_options = man_prog_info.get_options()
man_prog_attach_types = man_prog_info.get_attach_types()
man_prog_info.close()
bashcomp_info.reset_read() # We stopped at map types, rewind
bashcomp_prog_attach_types = bashcomp_info.get_prog_attach_types()
verify(source_prog_attach_types, help_prog_attach_types,
f'Comparing {ProgFileExtractor.filename} (attach_type_strings) and {ProgFileExtractor.filename} (do_help() ATTACH_TYPE):')
verify(source_prog_attach_types, man_prog_attach_types,
f'Comparing {ProgFileExtractor.filename} (attach_type_strings) and {ManProgExtractor.filename} (ATTACH_TYPE):')
verify(help_prog_options, man_prog_options,
f'Comparing {ProgFileExtractor.filename} (do_help() OPTIONS) and {ManProgExtractor.filename} (OPTIONS):')
verify(source_prog_attach_types, bashcomp_prog_attach_types,
f'Comparing {ProgFileExtractor.filename} (attach_type_strings) and {BashcompExtractor.filename} (BPFTOOL_PROG_ATTACH_TYPES):')
# Cgroup attach types
source_cgroup_attach_types = set(common_info.get_cgroup_attach_types().values())
common_info.close()
cgroup_info = CgroupFileExtractor()
help_cgroup_attach_types = cgroup_info.get_prog_attach_help()
help_cgroup_options = cgroup_info.get_options()
cgroup_info.close()
man_cgroup_info = ManCgroupExtractor()
man_cgroup_options = man_cgroup_info.get_options()
man_cgroup_attach_types = man_cgroup_info.get_attach_types()
man_cgroup_info.close()
bashcomp_cgroup_attach_types = bashcomp_info.get_cgroup_attach_types()
bashcomp_info.close()
verify(source_cgroup_attach_types, help_cgroup_attach_types,
f'Comparing {CommonFileExtractor.filename} (attach_type_strings) and {CgroupFileExtractor.filename} (do_help() ATTACH_TYPE):')
verify(source_cgroup_attach_types, man_cgroup_attach_types,
f'Comparing {CommonFileExtractor.filename} (attach_type_strings) and {ManCgroupExtractor.filename} (ATTACH_TYPE):')
verify(help_cgroup_options, man_cgroup_options,
f'Comparing {CgroupFileExtractor.filename} (do_help() OPTIONS) and {ManCgroupExtractor.filename} (OPTIONS):')
verify(source_cgroup_attach_types, bashcomp_cgroup_attach_types,
f'Comparing {CommonFileExtractor.filename} (attach_type_strings) and {BashcompExtractor.filename} (BPFTOOL_CGROUP_ATTACH_TYPES):')
# Options for remaining commands
for cmd in [ 'btf', 'feature', 'gen', 'iter', 'link', 'net', 'perf', 'struct_ops', ]:
source_info = GenericSourceExtractor(cmd + '.c')
help_cmd_options = source_info.get_options()
source_info.close()
man_cmd_info = ManGenericExtractor(os.path.join('Documentation', 'bpftool-' + cmd + '.rst'))
man_cmd_options = man_cmd_info.get_options()
man_cmd_info.close()
verify(help_cmd_options, man_cmd_options,
f'Comparing {source_info.filename} (do_help() OPTIONS) and {man_cmd_info.filename} (OPTIONS):')
source_main_info = GenericSourceExtractor('main.c')
help_main_options = source_main_info.get_options()
source_main_info.close()
man_main_info = ManGenericExtractor(os.path.join('Documentation', 'bpftool.rst'))
man_main_options = man_main_info.get_options()
man_main_info.close()
verify(help_main_options, man_main_options,
f'Comparing {source_main_info.filename} (do_help() OPTIONS) and {man_main_info.filename} (OPTIONS):')
sys.exit(retval)
if __name__ == "__main__":
main()
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7