1. 07 Mar, 2023 14 commits
    • Yafang Shao's avatar
      bpf: bpf_struct_ops memory usage · f062226d
      Yafang Shao authored
      A new helper is introduced to calculate bpf_struct_ops memory usage.
      
      The result as follows,
      
      - before
      1: struct_ops  name count_map  flags 0x0
              key 4B  value 256B  max_entries 1  memlock 4096B
              btf_id 73
      
      - after
      1: struct_ops  name count_map  flags 0x0
              key 4B  value 256B  max_entries 1  memlock 5016B
              btf_id 73
      Signed-off-by: default avatarYafang Shao <laoar.shao@gmail.com>
      Link: https://lore.kernel.org/r/20230305124615.12358-13-laoar.shao@gmail.comSigned-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      f062226d
    • Yafang Shao's avatar
      bpf: queue_stack_maps memory usage · c6e66b42
      Yafang Shao authored
      A new helper is introduced to calculate queue_stack_maps memory usage.
      
      The result as follows,
      
      - before
      20: queue  name count_map  flags 0x0
              key 0B  value 4B  max_entries 65536  memlock 266240B
      21: stack  name count_map  flags 0x0
              key 0B  value 4B  max_entries 65536  memlock 266240B
      
      - after
      20: queue  name count_map  flags 0x0
              key 0B  value 4B  max_entries 65536  memlock 524288B
      21: stack  name count_map  flags 0x0
              key 0B  value 4B  max_entries 65536  memlock 524288B
      Signed-off-by: default avatarYafang Shao <laoar.shao@gmail.com>
      Link: https://lore.kernel.org/r/20230305124615.12358-12-laoar.shao@gmail.comSigned-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      c6e66b42
    • Yafang Shao's avatar
      bpf: devmap memory usage · fa5e83df
      Yafang Shao authored
      A new helper is introduced to calculate the memory usage of devmap and
      devmap_hash. The number of dynamically allocated elements are recored
      for devmap_hash already, but not for devmap. To track the memory size of
      dynamically allocated elements, this patch also count the numbers for
      devmap.
      
      The result as follows,
      - before
      40: devmap  name count_map  flags 0x80
              key 4B  value 4B  max_entries 65536  memlock 524288B
      41: devmap_hash  name count_map  flags 0x80
              key 4B  value 4B  max_entries 65536  memlock 524288B
      
      - after
      40: devmap  name count_map  flags 0x80  <<<< no elements
              key 4B  value 4B  max_entries 65536  memlock 524608B
      41: devmap_hash  name count_map  flags 0x80 <<<< no elements
              key 4B  value 4B  max_entries 65536  memlock 524608B
      
      Note that the number of buckets is same with max_entries for devmap_hash
      in this case.
      Signed-off-by: default avatarYafang Shao <laoar.shao@gmail.com>
      Link: https://lore.kernel.org/r/20230305124615.12358-11-laoar.shao@gmail.comSigned-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      fa5e83df
    • Yafang Shao's avatar
      bpf: cpumap memory usage · 835f1fca
      Yafang Shao authored
      A new helper is introduced to calculate cpumap memory usage. The size of
      cpu_entries can be dynamically changed when we update or delete a cpumap
      element, but this patch doesn't include the memory size of cpu_entry
      yet. We can dynamically calculate the memory usage when we alloc or free
      a cpu_entry, but it will take extra runtime overhead, so let just put it
      aside currently. Note that the size of different cpu_entry may be
      different as well.
      
      The result as follows,
      - before
      48: cpumap  name count_map  flags 0x4
              key 4B  value 4B  max_entries 64  memlock 4096B
      
      - after
      48: cpumap  name count_map  flags 0x4
              key 4B  value 4B  max_entries 64  memlock 832B
      Signed-off-by: default avatarYafang Shao <laoar.shao@gmail.com>
      Link: https://lore.kernel.org/r/20230305124615.12358-10-laoar.shao@gmail.comSigned-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      835f1fca
    • Yafang Shao's avatar
      bpf: bloom_filter memory usage · 71a49abe
      Yafang Shao authored
      Introduce a new helper to calculate the bloom_filter memory usage.
      
      The result as follows,
      - before
      16: bloom_filter  flags 0x0
              key 0B  value 8B  max_entries 65536  memlock 524288B
      
      - after
      16: bloom_filter  flags 0x0
              key 0B  value 8B  max_entries 65536  memlock 65856B
      Signed-off-by: default avatarYafang Shao <laoar.shao@gmail.com>
      Link: https://lore.kernel.org/r/20230305124615.12358-9-laoar.shao@gmail.comSigned-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      71a49abe
    • Yafang Shao's avatar
      bpf: ringbuf memory usage · 2f7e4ab2
      Yafang Shao authored
      A new helper ringbuf_map_mem_usage() is introduced to calculate ringbuf
      memory usage.
      
      The result as follows,
      - before
      15: ringbuf  name count_map  flags 0x0
              key 0B  value 0B  max_entries 65536  memlock 0B
      
      - after
      15: ringbuf  name count_map  flags 0x0
              key 0B  value 0B  max_entries 65536  memlock 78424B
      Signed-off-by: default avatarYafang Shao <laoar.shao@gmail.com>
      Acked-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/r/20230305124615.12358-8-laoar.shao@gmail.comSigned-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      2f7e4ab2
    • Yafang Shao's avatar
      bpf: reuseport_array memory usage · 2e89caf0
      Yafang Shao authored
      A new helper is introduced to calculate reuseport_array memory usage.
      
      The result as follows,
      - before
      14: reuseport_sockarray  name count_map  flags 0x0
              key 4B  value 8B  max_entries 65536  memlock 1048576B
      
      - after
      14: reuseport_sockarray  name count_map  flags 0x0
              key 4B  value 8B  max_entries 65536  memlock 524544B
      Signed-off-by: default avatarYafang Shao <laoar.shao@gmail.com>
      Link: https://lore.kernel.org/r/20230305124615.12358-7-laoar.shao@gmail.comSigned-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      2e89caf0
    • Yafang Shao's avatar
      bpf: stackmap memory usage · cbb9b606
      Yafang Shao authored
      A new helper is introduced to get stackmap memory usage. Some small
      memory allocations are ignored as their memory size is quite small
      compared to the totol usage.
      
      The result as follows,
      - before
      16: stack_trace  name count_map  flags 0x0
              key 4B  value 8B  max_entries 65536  memlock 1048576B
      
      - after
      16: stack_trace  name count_map  flags 0x0
              key 4B  value 8B  max_entries 65536  memlock 2097472B
      Signed-off-by: default avatarYafang Shao <laoar.shao@gmail.com>
      Link: https://lore.kernel.org/r/20230305124615.12358-6-laoar.shao@gmail.comSigned-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      cbb9b606
    • Yafang Shao's avatar
      bpf: arraymap memory usage · 1746d055
      Yafang Shao authored
      Introduce array_map_mem_usage() to calculate arraymap memory usage. In
      this helper, some small memory allocations are ignored, like the
      allocation of struct bpf_array_aux in prog_array. The inner_map_meta in
      array_of_map is also ignored.
      
      The result as follows,
      
      - before
      11: array  name count_map  flags 0x0
              key 4B  value 4B  max_entries 65536  memlock 524288B
      12: percpu_array  name count_map  flags 0x0
              key 4B  value 4B  max_entries 65536  memlock 8912896B
      13: perf_event_array  name count_map  flags 0x0
              key 4B  value 4B  max_entries 65536  memlock 524288B
      14: prog_array  name count_map  flags 0x0
              key 4B  value 4B  max_entries 65536  memlock 524288B
      15: cgroup_array  name count_map  flags 0x0
              key 4B  value 4B  max_entries 65536  memlock 524288B
      
      - after
      11: array  name count_map  flags 0x0
              key 4B  value 4B  max_entries 65536  memlock 524608B
      12: percpu_array  name count_map  flags 0x0
              key 4B  value 4B  max_entries 65536  memlock 17301824B
      13: perf_event_array  name count_map  flags 0x0
              key 4B  value 4B  max_entries 65536  memlock 524608B
      14: prog_array  name count_map  flags 0x0
              key 4B  value 4B  max_entries 65536  memlock 524608B
      15: cgroup_array  name count_map  flags 0x0
              key 4B  value 4B  max_entries 65536  memlock 524608B
      Signed-off-by: default avatarYafang Shao <laoar.shao@gmail.com>
      Link: https://lore.kernel.org/r/20230305124615.12358-5-laoar.shao@gmail.comSigned-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      1746d055
    • Yafang Shao's avatar
      bpf: hashtab memory usage · 304849a2
      Yafang Shao authored
      htab_map_mem_usage() is introduced to calculate hashmap memory usage. In
      this helper, some small memory allocations are ignore, as their size is
      quite small compared with the total size. The inner_map_meta in
      hash_of_map is also ignored.
      
      The result for hashtab as follows,
      
      - before this change
      1: hash  name count_map  flags 0x1  <<<< no prealloc, fully set
              key 16B  value 24B  max_entries 1048576  memlock 41943040B
      2: hash  name count_map  flags 0x1  <<<< no prealloc, none set
              key 16B  value 24B  max_entries 1048576  memlock 41943040B
      3: hash  name count_map  flags 0x0  <<<< prealloc
              key 16B  value 24B  max_entries 1048576  memlock 41943040B
      
      The memlock is always a fixed size whatever it is preallocated or
      not, and whatever the count of allocated elements is.
      
      - after this change
      1: hash  name count_map  flags 0x1    <<<< non prealloc, fully set
              key 16B  value 24B  max_entries 1048576  memlock 117441536B
      2: hash  name count_map  flags 0x1    <<<< non prealloc, non set
              key 16B  value 24B  max_entries 1048576  memlock 16778240B
      3: hash  name count_map  flags 0x0    <<<< prealloc
              key 16B  value 24B  max_entries 1048576  memlock 109056000B
      
      The memlock now is hashtab actually allocated.
      
      The result for percpu hash map as follows,
      - before this change
      4: percpu_hash  name count_map  flags 0x0       <<<< prealloc
              key 16B  value 24B  max_entries 1048576  memlock 822083584B
      5: percpu_hash  name count_map  flags 0x1       <<<< no prealloc
              key 16B  value 24B  max_entries 1048576  memlock 822083584B
      
      - after this change
      4: percpu_hash  name count_map  flags 0x0
              key 16B  value 24B  max_entries 1048576  memlock 897582080B
      5: percpu_hash  name count_map  flags 0x1
              key 16B  value 24B  max_entries 1048576  memlock 922748736B
      
      At worst, the difference can be 10x, for example,
      - before this change
      6: hash  name count_map  flags 0x0
              key 4B  value 4B  max_entries 1048576  memlock 8388608B
      
      - after this change
      6: hash  name count_map  flags 0x0
              key 4B  value 4B  max_entries 1048576  memlock 83889408B
      Signed-off-by: default avatarYafang Shao <laoar.shao@gmail.com>
      Acked-by: default avatarHou Tao <houtao1@huawei.com>
      Link: https://lore.kernel.org/r/20230305124615.12358-4-laoar.shao@gmail.comSigned-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      304849a2
    • Yafang Shao's avatar
      bpf: lpm_trie memory usage · 41d5941e
      Yafang Shao authored
      trie_mem_usage() is introduced to calculate the lpm_trie memory usage.
      Some small memory allocations are ignored. The inner node is also
      ignored.
      
      The result as follows,
      
      - before
      10: lpm_trie  flags 0x1
              key 8B  value 8B  max_entries 65536  memlock 1048576B
      
      - after
      10: lpm_trie  flags 0x1
              key 8B  value 8B  max_entries 65536  memlock 2291536B
      Signed-off-by: default avatarYafang Shao <laoar.shao@gmail.com>
      Link: https://lore.kernel.org/r/20230305124615.12358-3-laoar.shao@gmail.comSigned-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      41d5941e
    • Yafang Shao's avatar
      bpf: add new map ops ->map_mem_usage · 90a5527d
      Yafang Shao authored
      Add a new map ops ->map_mem_usage to print the memory usage of a
      bpf map.
      
      This is a preparation for the followup change.
      Signed-off-by: default avatarYafang Shao <laoar.shao@gmail.com>
      Link: https://lore.kernel.org/r/20230305124615.12358-2-laoar.shao@gmail.comSigned-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      90a5527d
    • Nathan Chancellor's avatar
      bpf: Increase size of BTF_ID_LIST without CONFIG_DEBUG_INFO_BTF again · 2d5bcdcd
      Nathan Chancellor authored
      After commit 66e3a13e ("bpf: Add bpf_dynptr_slice and bpf_dynptr_slice_rdwr"),
      clang builds without CONFIG_DEBUG_INFO_BTF warn:
      
        kernel/bpf/verifier.c:10298:24: warning: array index 16 is past the end of the array (that has type 'u32[16]' (aka 'unsigned int[16]')) [-Warray-bounds]
                                           meta.func_id == special_kfunc_list[KF_bpf_dynptr_slice_rdwr]) {
                                                           ^                  ~~~~~~~~~~~~~~~~~~~~~~~~
        kernel/bpf/verifier.c:9150:1: note: array 'special_kfunc_list' declared here
        BTF_ID_LIST(special_kfunc_list)
        ^
        include/linux/btf_ids.h:207:27: note: expanded from macro 'BTF_ID_LIST'
        #define BTF_ID_LIST(name) static u32 __maybe_unused name[16];
                                  ^
        1 warning generated.
      
      A warning of this nature was previously addressed by
      commit beb3d47d ("bpf: Fix a BTF_ID_LIST bug with CONFIG_DEBUG_INFO_BTF not set")
      but there have been new kfuncs added since then.
      
      Quadruple the size of the CONFIG_DEBUG_INFO_BTF=n definition so that
      this problem is unlikely to show up for some time.
      
      Link: https://github.com/ClangBuiltLinux/linux/issues/1810Signed-off-by: default avatarNathan Chancellor <nathan@kernel.org>
      Reviewed-by: default avatarTom Rix <trix@redhat.com>
      Link: https://lore.kernel.org/r/20230307-bpf-kfuncs-warray-bounds-v1-1-00ad3191f3a6@kernel.orgSigned-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      2d5bcdcd
    • Jakub Kicinski's avatar
      Merge tag 'for-netdev' of https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next · 36e5e391
      Jakub Kicinski authored
      Daniel Borkmann says:
      
      ====================
      pull-request: bpf-next 2023-03-06
      
      We've added 85 non-merge commits during the last 13 day(s) which contain
      a total of 131 files changed, 7102 insertions(+), 1792 deletions(-).
      
      The main changes are:
      
      1) Add skb and XDP typed dynptrs which allow BPF programs for more
         ergonomic and less brittle iteration through data and variable-sized
         accesses, from Joanne Koong.
      
      2) Bigger batch of BPF verifier improvements to prepare for upcoming BPF
         open-coded iterators allowing for less restrictive looping capabilities,
         from Andrii Nakryiko.
      
      3) Rework RCU enforcement in the verifier, add kptr_rcu and enforce BPF
         programs to NULL-check before passing such pointers into kfunc,
         from Alexei Starovoitov.
      
      4) Add support for kptrs in percpu hashmaps, percpu LRU hashmaps and in
         local storage maps, from Kumar Kartikeya Dwivedi.
      
      5) Add BPF verifier support for ST instructions in convert_ctx_access()
         which will help new -mcpu=v4 clang flag to start emitting them,
         from Eduard Zingerman.
      
      6) Make uprobe attachment Android APK aware by supporting attachment
         to functions inside ELF objects contained in APKs via function names,
         from Daniel Müller.
      
      7) Add a new flag BPF_F_TIMER_ABS flag for bpf_timer_start() helper
         to start the timer with absolute expiration value instead of relative
         one, from Tero Kristo.
      
      8) Add a new kfunc bpf_cgroup_from_id() to look up cgroups via id,
         from Tejun Heo.
      
      9) Extend libbpf to support users manually attaching kprobes/uprobes
         in the legacy/perf/link mode, from Menglong Dong.
      
      10) Implement workarounds in the mips BPF JIT for DADDI/R4000,
         from Jiaxun Yang.
      
      11) Enable mixing bpf2bpf and tailcalls for the loongarch BPF JIT,
          from Hengqi Chen.
      
      12) Extend BPF instruction set doc with describing the encoding of BPF
          instructions in terms of how bytes are stored under big/little endian,
          from Jose E. Marchesi.
      
      13) Follow-up to enable kfunc support for riscv BPF JIT, from Pu Lehui.
      
      14) Fix bpf_xdp_query() backwards compatibility on old kernels,
          from Yonghong Song.
      
      15) Fix BPF selftest cross compilation with CLANG_CROSS_FLAGS,
          from Florent Revest.
      
      16) Improve bpf_cpumask_ma to only allocate one bpf_mem_cache,
          from Hou Tao.
      
      17) Fix BPF verifier's check_subprogs to not unnecessarily mark
          a subprogram with has_tail_call, from Ilya Leoshkevich.
      
      18) Fix arm syscall regs spec in libbpf's bpf_tracing.h, from Puranjay Mohan.
      
      * tag 'for-netdev' of https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next: (85 commits)
        selftests/bpf: Add test for legacy/perf kprobe/uprobe attach mode
        selftests/bpf: Split test_attach_probe into multi subtests
        libbpf: Add support to set kprobe/uprobe attach mode
        tools/resolve_btfids: Add /libsubcmd to .gitignore
        bpf: add support for fixed-size memory pointer returns for kfuncs
        bpf: generalize dynptr_get_spi to be usable for iters
        bpf: mark PTR_TO_MEM as non-null register type
        bpf: move kfunc_call_arg_meta higher in the file
        bpf: ensure that r0 is marked scratched after any function call
        bpf: fix visit_insn()'s detection of BPF_FUNC_timer_set_callback helper
        bpf: clean up visit_insn()'s instruction processing
        selftests/bpf: adjust log_fixup's buffer size for proper truncation
        bpf: honor env->test_state_freq flag in is_state_visited()
        selftests/bpf: enhance align selftest's expected log matching
        bpf: improve regsafe() checks for PTR_TO_{MEM,BUF,TP_BUFFER}
        bpf: improve stack slot state printing
        selftests/bpf: Disassembler tests for verifier.c:convert_ctx_access()
        selftests/bpf: test if pointer type is tracked for BPF_ST_MEM
        bpf: allow ctx writes using BPF_ST_MEM instruction
        bpf: Use separate RCU callbacks for freeing selem
        ...
      ====================
      
      Link: https://lore.kernel.org/r/20230307004346.27578-1-daniel@iogearbox.netSigned-off-by: default avatarJakub Kicinski <kuba@kernel.org>
      36e5e391
  2. 06 Mar, 2023 5 commits
  3. 04 Mar, 2023 16 commits
  4. 03 Mar, 2023 5 commits
    • Kumar Kartikeya Dwivedi's avatar
      bpf: Use separate RCU callbacks for freeing selem · e768e3c5
      Kumar Kartikeya Dwivedi authored
      Martin suggested that instead of using a byte in the hole (which he has
      a use for in his future patch) in bpf_local_storage_elem, we can
      dispatch a different call_rcu callback based on whether we need to free
      special fields in bpf_local_storage_elem data. The free path, described
      in commit 9db44fdd ("bpf: Support kptrs in local storage maps"),
      only waits for call_rcu callbacks when there are special (kptrs, etc.)
      fields in the map value, hence it is necessary that we only access
      smap in this case.
      
      Therefore, dispatch different RCU callbacks based on the BPF map has a
      valid btf_record, which dereference and use smap's btf_record only when
      it is valid.
      Signed-off-by: default avatarKumar Kartikeya Dwivedi <memxor@gmail.com>
      Link: https://lore.kernel.org/r/20230303141542.300068-1-memxor@gmail.comSigned-off-by: default avatarMartin KaFai Lau <martin.lau@kernel.org>
      e768e3c5
    • Daniel Borkmann's avatar
      Merge branch 'bpf-kptr-rcu' · db55174d
      Daniel Borkmann authored
      Alexei Starovoitov says:
      
      ====================
      v4->v5:
      fix typos, add acks.
      
      v3->v4:
      - patch 3 got much cleaner after BPF_KPTR_RCU was removed as suggested by David.
      
      - make KF_RCU stronger and require that bpf program checks for NULL
      before passing such pointers into kfunc. The prog has to do that anyway
      to access fields and it aligns with BTF_TYPE_SAFE_RCU allowlist.
      
      - New patch 6: refactor RCU enforcement in the verifier.
      The patches 2,3,6 are part of one feature.
      The 2 and 3 alone are incomplete, since RCU pointers are barely useful
      without bpf_rcu_read_lock/unlock in GCC compiled kernel.
      Even if GCC lands support for btf_type_tag today it will take time
      to mandate that version for kernel builds. Hence go with allow list
      approach. See patch 6 for details.
      This allows to start strict enforcement of TRUSTED | UNTRUSTED
      in one part of PTR_TO_BTF_ID accesses.
      One step closer to KF_TRUSTED_ARGS by default.
      
      v2->v3:
      - Instead of requiring bpf progs to tag fields with __kptr_rcu
      teach the verifier to infer RCU properties based on the type.
      BPF_KPTR_RCU becomes kernel internal type of struct btf_field.
      - Add patch 2 to tag cgroups and dfl_cgrp as trusted.
      That bug was spotted by BPF CI on clang compiler kernels,
      since patch 3 is doing:
      static bool in_rcu_cs(struct bpf_verifier_env *env)
      {
              return env->cur_state->active_rcu_lock || !env->prog->aux->sleepable;
      }
      which makes all non-sleepable programs behave like they have implicit
      rcu_read_lock around them. Which is the case in practice.
      It was fine on gcc compiled kernels where task->cgroup deference was producing
      PTR_TO_BTF_ID, but on clang compiled kernels task->cgroup deference was
      producing PTR_TO_BTF_ID | MEM_RCU | MAYBE_NULL, which is more correct,
      but selftests were failing. Patch 2 fixes this discrepancy.
      With few more patches like patch 2 we can make KF_TRUSTED_ARGS default
      for kfuncs and helpers.
      - Add comment in selftest patch 5 that it's verifier only check.
      
      v1->v2:
      Instead of agressively allow dereferenced kptr_rcu pointers into KF_TRUSTED_ARGS
      kfuncs only allow them into KF_RCU funcs.
      The KF_RCU flag is a weaker version of KF_TRUSTED_ARGS. The kfuncs marked with
      KF_RCU expect either PTR_TRUSTED or MEM_RCU arguments. The verifier guarantees
      that the objects are valid and there is no use-after-free, but the pointers
      maybe NULL and pointee object's reference count could have reached zero, hence
      kfuncs must do != NULL check and consider refcnt==0 case when accessing such
      arguments.
      No changes in patch 1.
      Patches 2,3,4 adjusted with above behavior.
      
      v1:
      The __kptr_ref turned out to be too limited, since any "trusted" pointer access
      requires bpf_kptr_xchg() which is impractical when the same pointer needs
      to be dereferenced by multiple cpus.
      The __kptr "untrusted" only access isn't very useful in practice.
      Rename __kptr to __kptr_untrusted with eventual goal to deprecate it,
      and rename __kptr_ref to __kptr, since that looks to be more common use of kptrs.
      Introduce __kptr_rcu that can be directly dereferenced and used similar
      to native kernel C code.
      Once bpf_cpumask and task_struct kfuncs are converted to observe RCU GP
      when refcnt goes to zero, both __kptr and __kptr_untrusted can be deprecated
      and __kptr_rcu can become the only __kptr tag.
      ====================
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      db55174d
    • Alexei Starovoitov's avatar
      bpf: Refactor RCU enforcement in the verifier. · 6fcd486b
      Alexei Starovoitov authored
      bpf_rcu_read_lock/unlock() are only available in clang compiled kernels. Lack
      of such key mechanism makes it impossible for sleepable bpf programs to use RCU
      pointers.
      
      Allow bpf_rcu_read_lock/unlock() in GCC compiled kernels (though GCC doesn't
      support btf_type_tag yet) and allowlist certain field dereferences in important
      data structures like tast_struct, cgroup, socket that are used by sleepable
      programs either as RCU pointer or full trusted pointer (which is valid outside
      of RCU CS). Use BTF_TYPE_SAFE_RCU and BTF_TYPE_SAFE_TRUSTED macros for such
      tagging. They will be removed once GCC supports btf_type_tag.
      
      With that refactor check_ptr_to_btf_access(). Make it strict in enforcing
      PTR_TRUSTED and PTR_UNTRUSTED while deprecating old PTR_TO_BTF_ID without
      modifier flags. There is a chance that this strict enforcement might break
      existing programs (especially on GCC compiled kernels), but this cleanup has to
      start sooner than later. Note PTR_TO_CTX access still yields old deprecated
      PTR_TO_BTF_ID. Once it's converted to strict PTR_TRUSTED or PTR_UNTRUSTED the
      kfuncs and helpers will be able to default to KF_TRUSTED_ARGS. KF_RCU will
      remain as a weaker version of KF_TRUSTED_ARGS where obj refcnt could be 0.
      
      Adjust rcu_read_lock selftest to run on gcc and clang compiled kernels.
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      Acked-by: default avatarDavid Vernet <void@manifault.com>
      Link: https://lore.kernel.org/bpf/20230303041446.3630-7-alexei.starovoitov@gmail.com
      6fcd486b
    • Alexei Starovoitov's avatar
      selftests/bpf: Tweak cgroup kfunc test. · 0047d834
      Alexei Starovoitov authored
      Adjust cgroup kfunc test to dereference RCU protected cgroup pointer
      as PTR_TRUSTED and pass into KF_TRUSTED_ARGS kfunc.
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      Acked-by: default avatarDavid Vernet <void@manifault.com>
      Link: https://lore.kernel.org/bpf/20230303041446.3630-6-alexei.starovoitov@gmail.com
      0047d834
    • Alexei Starovoitov's avatar