1. 20 Oct, 2021 8 commits
  2. 19 Oct, 2021 7 commits
    • Kumar Kartikeya Dwivedi's avatar
      bpf: Silence Coverity warning for find_kfunc_desc_btf · 588cd7ef
      Kumar Kartikeya Dwivedi authored
      The helper function returns a pointer that in the failure case encodes
      an error in the struct btf pointer. The current code lead to Coverity
      warning about the use of the invalid pointer:
      
       *** CID 1507963:  Memory - illegal accesses  (USE_AFTER_FREE)
       /kernel/bpf/verifier.c: 1788 in find_kfunc_desc_btf()
       1782                          return ERR_PTR(-EINVAL);
       1783                  }
       1784
       1785                  kfunc_btf = __find_kfunc_desc_btf(env, offset, btf_modp);
       1786                  if (IS_ERR_OR_NULL(kfunc_btf)) {
       1787                          verbose(env, "cannot find module BTF for func_id %u\n", func_id);
       >>>      CID 1507963:  Memory - illegal accesses  (USE_AFTER_FREE)
       >>>      Using freed pointer "kfunc_btf".
       1788                          return kfunc_btf ?: ERR_PTR(-ENOENT);
       1789                  }
       1790                  return kfunc_btf;
       1791          }
       1792          return btf_vmlinux ?: ERR_PTR(-ENOENT);
       1793     }
      
      Daniel suggested the use of ERR_CAST so that the intended use is clear
      to Coverity, but on closer look it seems that we never return NULL from
      the helper. Andrii noted that since __find_kfunc_desc_btf already logs
      errors for all cases except btf_get_by_fd, it is much easier to add
      logging for that and remove the IS_ERR check altogether, returning
      directly from it.
      Suggested-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Signed-off-by: default avatarKumar Kartikeya Dwivedi <memxor@gmail.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20211009040900.803436-1-memxor@gmail.com
      588cd7ef
    • Andrii Nakryiko's avatar
      Merge branch 'fixes for bpftool's Makefile' · 32fa0efa
      Andrii Nakryiko authored
      Quentin Monnet says:
      
      ====================
      
      This set contains one fix for bpftool's Makefile, to make sure that the
      headers internal to libbpf are installed properly even if we add more
      headers to the relevant Makefile variable in the future (although we'd like
      to avoid that if possible).
      
      The other patches aim at cleaning up the output from the Makefile, in
      particular when running the command "make" another time after bpftool is
      built.
      ====================
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      32fa0efa
    • Quentin Monnet's avatar
      bpftool: Turn check on zlib from a phony target into a conditional error · 062e1fc0
      Quentin Monnet authored
      One of bpftool's object files depends on zlib. To make sure we do not
      attempt to build that object when the library is not available, commit
      d66fa3c7 ("tools: bpftool: add feature check for zlib") introduced a
      feature check to detect whether zlib is present.
      
      This check comes as a rule for which the target ("zdep") is a
      nonexistent file (phony target), which means that the Makefile always
      attempts to rebuild it. It is mostly harmless. However, one side effect
      is that, on running again once bpftool is already built, make considers
      that "something" (the recipe for zdep) was executed, and does not print
      the usual message "make: Nothing to be done for 'all'", which is a
      user-friendly indicator that the build went fine.
      
      Before, with some level of debugging information:
      
          $ make --debug=m
          [...]
          Reading makefiles...
      
          Auto-detecting system features:
          ...                        libbfd: [ on  ]
          ...        disassembler-four-args: [ on  ]
          ...                          zlib: [ on  ]
          ...                        libcap: [ on  ]
          ...               clang-bpf-co-re: [ on  ]
      
          Updating makefiles....
          Updating goal targets....
           File 'all' does not exist.
                 File 'zdep' does not exist.
                Must remake target 'zdep'.
           File 'all' does not exist.
          Must remake target 'all'.
          Successfully remade target file 'all'.
      
      After the patch:
      
          $ make --debug=m
          [...]
      
          Auto-detecting system features:
          ...                        libbfd: [ on  ]
          ...        disassembler-four-args: [ on  ]
          ...                          zlib: [ on  ]
          ...                        libcap: [ on  ]
          ...               clang-bpf-co-re: [ on  ]
      
          Updating makefiles....
          Updating goal targets....
           File 'all' does not exist.
          Must remake target 'all'.
          Successfully remade target file 'all'.
          make: Nothing to be done for 'all'.
      
      (Note the last line, which is not part of make's debug information.)
      Signed-off-by: default avatarQuentin Monnet <quentin@isovalent.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Acked-by: default avatarJohn Fastabend <john.fastabend@gmail.com>
      Link: https://lore.kernel.org/bpf/20211009210341.6291-4-quentin@isovalent.com
      062e1fc0
    • Quentin Monnet's avatar
      bpftool: Do not FORCE-build libbpf · ced846c6
      Quentin Monnet authored
      In bpftool's Makefile, libbpf has a FORCE dependency, to make sure we
      rebuild it in case its source files changed. Let's instead make the
      rebuild depend on the source files directly, through a call to the
      "$(wildcard ...)" function. This avoids descending into libbpf's
      directory if there is nothing to update.
      
      Do the same for the bootstrap libbpf version.
      
      This results in a slightly faster operation and less verbose output when
      running make a second time in bpftool's directory.
      
      Before:
      
          Auto-detecting system features:
          ...                        libbfd: [ on  ]
          ...        disassembler-four-args: [ on  ]
          ...                          zlib: [ on  ]
          ...                        libcap: [ on  ]
          ...               clang-bpf-co-re: [ on  ]
      
          make[1]: Entering directory '/root/dev/linux/tools/lib/bpf'
          make[1]: Entering directory '/root/dev/linux/tools/lib/bpf'
          make[1]: Nothing to be done for 'install_headers'.
          make[1]: Leaving directory '/root/dev/linux/tools/lib/bpf'
          make[1]: Leaving directory '/root/dev/linux/tools/lib/bpf'
      
      After:
      
          Auto-detecting system features:
          ...                        libbfd: [ on  ]
          ...        disassembler-four-args: [ on  ]
          ...                          zlib: [ on  ]
          ...                        libcap: [ on  ]
          ...               clang-bpf-co-re: [ on  ]
      
      Other ways to clean up the output could be to pass the "-s" option, or
      to redirect the output to >/dev/null, when calling make recursively to
      descend into libbpf's directory. However, this would suppress some
      useful output if something goes wrong during the build. A better
      alternative would be to pass "--no-print-directory" to the recursive
      make, but that would still leave us with some noise for
      "install_headers". Skipping the descent into libbpf's directory if no
      source file has changed works best, and seems the most logical option
      overall.
      Reported-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Signed-off-by: default avatarQuentin Monnet <quentin@isovalent.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Acked-by: default avatarJohn Fastabend <john.fastabend@gmail.com>
      Link: https://lore.kernel.org/bpf/20211009210341.6291-3-quentin@isovalent.com
      ced846c6
    • Quentin Monnet's avatar
      bpftool: Fix install for libbpf's internal header(s) · 34e3ab14
      Quentin Monnet authored
      We recently updated bpftool's Makefile to make it install the headers
      from libbpf, instead of pulling them directly from libbpf's directory.
      There is also an additional header, internal to libbpf, that needs be
      installed. The way that bpftool's Makefile installs that particular
      header is currently correct, but would break if we were to modify
      $(LIBBPF_INTERNAL_HDRS) to make it point to more than one header.
      
      Use a static pattern rule instead, so that the Makefile can withstand
      the addition of other headers to install.
      
      The objective is simply to make the Makefile more robust. It should
      _not_ be read as an invitation to import more internal headers from
      libbpf into bpftool.
      
      Fixes: f012ade1 ("bpftool: Install libbpf headers instead of including the dir")
      Reported-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Signed-off-by: default avatarQuentin Monnet <quentin@isovalent.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Acked-by: default avatarJohn Fastabend <john.fastabend@gmail.com>
      Link: https://lore.kernel.org/bpf/20211009210341.6291-2-quentin@isovalent.com
      34e3ab14
    • Quentin Monnet's avatar
      libbpf: Remove Makefile warnings on out-of-sync netlink.h/if_link.h · d51b6b22
      Quentin Monnet authored
      Although relying on some definitions from the netlink.h and if_link.h
      headers copied into tools/include/uapi/linux/, libbpf does not need
      those headers to stay entirely up-to-date with their original versions,
      and the warnings emitted by the Makefile when it detects a difference
      are usually just noise. Let's remove those warnings.
      Suggested-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Signed-off-by: default avatarQuentin Monnet <quentin@isovalent.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20211010002528.9772-1-quentin@isovalent.com
      d51b6b22
    • Yonghong Song's avatar
      bpf: Rename BTF_KIND_TAG to BTF_KIND_DECL_TAG · 223f903e
      Yonghong Song authored
      Patch set [1] introduced BTF_KIND_TAG to allow tagging
      declarations for struct/union, struct/union field, var, func
      and func arguments and these tags will be encoded into
      dwarf. They are also encoded to btf by llvm for the bpf target.
      
      After BTF_KIND_TAG is introduced, we intended to use it
      for kernel __user attributes. But kernel __user is actually
      a type attribute. Upstream and internal discussion showed
      it is not a good idea to mix declaration attribute and
      type attribute. So we proposed to introduce btf_type_tag
      as a type attribute and existing btf_tag renamed to
      btf_decl_tag ([2]).
      
      This patch renamed BTF_KIND_TAG to BTF_KIND_DECL_TAG and some
      other declarations with *_tag to *_decl_tag to make it clear
      the tag is for declaration. In the future, BTF_KIND_TYPE_TAG
      might be introduced per [3].
      
       [1] https://lore.kernel.org/bpf/20210914223004.244411-1-yhs@fb.com/
       [2] https://reviews.llvm.org/D111588
       [3] https://reviews.llvm.org/D111199
      
      Fixes: b5ea834d ("bpf: Support for new btf kind BTF_KIND_TAG")
      Fixes: 5b84bd10 ("libbpf: Add support for BTF_KIND_TAG")
      Fixes: 5c07f2fe ("bpftool: Add support for BTF_KIND_TAG")
      Signed-off-by: default avatarYonghong Song <yhs@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Link: https://lore.kernel.org/bpf/20211012164838.3345699-1-yhs@fb.com
      223f903e
  3. 11 Oct, 2021 2 commits
  4. 09 Oct, 2021 1 commit
  5. 08 Oct, 2021 22 commits
    • Andrii Nakryiko's avatar
      Merge branch 'selftests/bpf: Add parallelism to test_progs' · e52a8b96
      Andrii Nakryiko authored
      Yucong Sun says:
      
      ====================
      
      This patch series adds "-j" parelell execution to test_progs, with "--debug" to
      display server/worker communications. Also, some Tests that often fails in
      parallel are marked as serial test, and it will run in sequence after parallel
      execution is done.
      
      This patch series also adds a error summary after all tests execution finished.
      
      V6 -> V5:
        * adding error summary logic for non parallel mode too.
        * changed how serial tests are implemented, use main process instead of worker 0.
        * fixed a dozen broken test when running in parallel.
      
      V5 -> V4:
        * change to SOCK_SEQPACKET for close notification.
        * move all debug output to "--debug" mode
        * output log as test finish, and all error logs again after summary line.
        * variable naming / style changes
        * adds serial_test_name() to replace serial test lists.
      ====================
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      e52a8b96
    • Yucong Sun's avatar
      selfetest/bpf: Make some tests serial · d3f7b166
      Yucong Sun authored
      Change tests that often fails in parallel execution mode to serial.
      Signed-off-by: default avatarYucong Sun <sunyucong@gmail.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20211006185619.364369-15-fallentree@fb.com
      d3f7b166
    • Yucong Sun's avatar
      selftests/bpf: Fix pid check in fexit_sleep test · 5db02dd7
      Yucong Sun authored
      bpf_get_current_pid_tgid() returns u64, whose upper 32 bits are the same
      as userspace getpid() return value.
      Signed-off-by: default avatarYucong Sun <sunyucong@gmail.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20211006185619.364369-13-fallentree@fb.com
      5db02dd7
    • Yucong Sun's avatar
      selftests/bpf: Adding pid filtering for atomics test · 0f4feacc
      Yucong Sun authored
      This make atomics test able to run in parallel with other tests.
      Signed-off-by: default avatarYucong Sun <sunyucong@gmail.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20211006185619.364369-11-fallentree@fb.com
      0f4feacc
    • Yucong Sun's avatar
      selftests/bpf: Make cgroup_v1v2 use its own port · 445e72c7
      Yucong Sun authored
      This patch change cgroup_v1v2 use a different port, avoid conflict with
      other tests.
      Signed-off-by: default avatarYucong Sun <sunyucong@gmail.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20211006185619.364369-8-fallentree@fb.com
      445e72c7
    • Yucong Sun's avatar
      selftests/bpf: Fix race condition in enable_stats · d719de0d
      Yucong Sun authored
      In parallel execution mode, this test now need to use atomic operation
      to avoid race condition.
      Signed-off-by: default avatarYucong Sun <sunyucong@gmail.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20211006185619.364369-7-fallentree@fb.com
      d719de0d
    • Yucong Sun's avatar
      selftests/bpf: Add per worker cgroup suffix · e87c3434
      Yucong Sun authored
      This patch make each worker use a unique cgroup base directory, thus
      allowing tests that uses cgroups to run concurrently.
      Signed-off-by: default avatarYucong Sun <sunyucong@gmail.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20211006185619.364369-5-fallentree@fb.com
      e87c3434
    • Yucong Sun's avatar
      selftests/bpf: Allow some tests to be executed in sequence · 6587ff58
      Yucong Sun authored
      This patch allows tests to define serial_test_name() instead of
      test_name(), and this will make test_progs execute those in sequence
      after all other tests finished executing concurrently.
      Signed-off-by: default avatarYucong Sun <sunyucong@gmail.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20211006185619.364369-3-fallentree@fb.com
      6587ff58
    • Yucong Sun's avatar
      selftests/bpf: Add parallelism to test_progs · 91b2c0af
      Yucong Sun authored
      This patch adds "-j" mode to test_progs, executing tests in multiple
      process.  "-j" mode is optional, and works with all existing test
      selection mechanism, as well as "-v", "-l" etc.
      
      In "-j" mode, main process use UDS/SEQPACKET to communicate to each forked
      worker, commanding it to run tests and collect logs. After all tests are
      finished, a summary is printed. main process use multiple competing
      threads to dispatch work to worker, trying to keep them all busy.
      
      The test status will be printed as soon as it is finished, if there are
      error logs, it will be printed after the final summary line.
      
      By specifying "--debug", additional debug information on server/worker
      communication will be printed.
      
      Example output:
        > ./test_progs -n 15-20 -j
        [   12.801730] bpf_testmod: loading out-of-tree module taints kernel.
        Launching 8 workers.
        #20 btf_split:OK
        #16 btf_endian:OK
        #18 btf_module:OK
        #17 btf_map_in_map:OK
        #19 btf_skc_cls_ingress:OK
        #15 btf_dump:OK
        Summary: 6/20 PASSED, 0 SKIPPED, 0 FAILED
      Signed-off-by: default avatarYucong Sun <sunyucong@gmail.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20211006185619.364369-2-fallentree@fb.com
      91b2c0af
    • Andrii Nakryiko's avatar
      Merge branch 'add support for writable bare tracepoint' · a1852ce0
      Andrii Nakryiko authored
      Hou Tao says:
      
      ====================
      
      From: Hou Tao <houtao1@huawei.com>
      
      Hi,
      
      The patchset series supports writable context for bare tracepoint.
      
      The main idea comes from patchset "writable contexts for bpf raw
      tracepoints" [1], but it only supports normal tracepoint with
      associated trace event under tracefs. Now we have one use case
      in which we add bare tracepoint in VFS layer, and update
      file::f_mode for specific files. The reason using bare tracepoint
      is that it doesn't form a ABI and we can change it freely. So
      add support for it in BPF.
      
      Comments are always welcome.
      
      [1]: https://lore.kernel.org/lkml/20190426184951.21812-1-mmullins@fb.com
      
      Change log:
      v5:
       * rebased on bpf-next
       * patch 1: add Acked-by tag
       * patch 2: handle invalid section name, make prefixes array being const
      
      v4: https://www.spinics.net/lists/bpf/msg47021.html
       * rebased on bpf-next
       * update patch 2 to add support for writable raw tracepoint attachment
         in attach_raw_tp().
       * update patch 3 to add Acked-by tag
      
      v3: https://www.spinics.net/lists/bpf/msg46824.html
        * use raw_tp.w instead of raw_tp_writable as section
          name of writable tp
        * use ASSERT_XXX() instead of CHECK()
        * define a common macro for "/sys/kernel/bpf_testmod"
      
      v2: https://www.spinics.net/lists/bpf/msg46356.html
        * rebase on bpf-next tree
        * address comments from Yonghong Song
        * rename bpf_testmode_test_writable_ctx::ret as early_ret to reflect
          its purpose better.
      
      v1: https://www.spinics.net/lists/bpf/msg46221.html
      ====================
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      a1852ce0
    • Hou Tao's avatar
      bpf/selftests: Add test for writable bare tracepoint · fa7f17d0
      Hou Tao authored
      Add a writable bare tracepoint in bpf_testmod module, and
      trigger its calling when reading /sys/kernel/bpf_testmod
      with a specific buffer length. The reading will return
      the value in writable context if the early return flag
      is enabled in writable context.
      Signed-off-by: default avatarHou Tao <houtao1@huawei.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Acked-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20211004094857.30868-4-hotforest@gmail.com
      fa7f17d0
    • Hou Tao's avatar
      libbpf: Support detecting and attaching of writable tracepoint program · ccaf12d6
      Hou Tao authored
      Program on writable tracepoint is BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE,
      but its attachment is the same as BPF_PROG_TYPE_RAW_TRACEPOINT.
      Signed-off-by: default avatarHou Tao <houtao1@huawei.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20211004094857.30868-3-hotforest@gmail.com
      ccaf12d6
    • Hou Tao's avatar
      bpf: Support writable context for bare tracepoint · 65223741
      Hou Tao authored
      Commit 9df1c28b ("bpf: add writable context for raw tracepoints")
      supports writable context for tracepoint, but it misses the support
      for bare tracepoint which has no associated trace event.
      
      Bare tracepoint is defined by DECLARE_TRACE(), so adding a corresponding
      DECLARE_TRACE_WRITABLE() macro to generate a definition in __bpf_raw_tp_map
      section for bare tracepoint in a similar way to DEFINE_TRACE_WRITABLE().
      Signed-off-by: default avatarHou Tao <houtao1@huawei.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Acked-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20211004094857.30868-2-hotforest@gmail.com
      65223741
    • Andrii Nakryiko's avatar
      Merge branch 'install libbpf headers when using the library' · 1c8dab7d
      Andrii Nakryiko authored
      Quentin Monnet says:
      
      ====================
      
      Libbpf is used at several locations in the repository. Most of the time,
      the tools relying on it build the library in its own directory, and include
      the headers from there. This works, but this is not the cleanest approach.
      It generates objects outside of the directory of the tool which is being
      built, and it also increases the risk that developers include a header file
      internal to libbpf, which is not supposed to be exposed to user
      applications.
      
      This set adjusts all involved Makefiles to make sure that libbpf is built
      locally (with respect to the tool's directory or provided build directory),
      and by ensuring that "make install_headers" is run from libbpf's Makefile
      to export user headers properly.
      
      This comes at a cost: given that the libbpf was so far mostly compiled in
      its own directory by the different components using it, compiling it once
      would be enough for all those components. With the new approach, each
      component compiles its own version. To mitigate this cost, efforts were
      made to reuse the compiled library when possible:
      
      - Make the bpftool version in samples/bpf reuse the library previously
        compiled for the selftests.
      - Make the bpftool version in BPF selftests reuse the library previously
        compiled for the selftests.
      - Similarly, make resolve_btfids in BPF selftests reuse the same compiled
        library.
      - Similarly, make runqslower in BPF selftests reuse the same compiled
        library; and make it rely on the bpftool version also compiled from the
        selftests (instead of compiling its own version).
      - runqslower, when compiled independently, needs its own version of
        bpftool: make them share the same compiled libbpf.
      
      As a result:
      
      - Compiling the samples/bpf should compile libbpf just once.
      - Compiling the BPF selftests should compile libbpf just once.
      - Compiling the kernel (with BTF support) should now lead to compiling
        libbpf twice: one for resolve_btfids, one for kernel/bpf/preload.
      - Compiling runqslower individually should compile libbpf just once. Same
        thing for bpftool, resolve_btfids, and kernel/bpf/preload/iterators.
      
      (Not accounting for the boostrap version of libbpf required by bpftool,
      which was already placed under a dedicated .../boostrap/libbpf/ directory,
      and for which the count remains unchanged.)
      
      A few commits in the series also contain drive-by clean-up changes for
      bpftool includes, samples/bpf/.gitignore, or test_bpftool_build.sh. Please
      refer to individual commit logs for details.
      
      v4:
        - Make the "libbpf_hdrs" dependency an order-only dependency in
          kernel/bpf/preload/Makefile, samples/bpf/Makefile, and
          tools/bpf/runqslower/Makefile. This is to avoid to unconditionally
          recompile the targets.
        - samples/bpf/.gitignore: prefix objects with a "/" to mark that we
          ignore them when at the root of the samples/bpf/ directory.
        - libbpf: add a commit to make "install_headers" depend on the header
          files, to avoid exporting again if the sources are older than the
          targets. This ensures that applications relying on those headers are
          not rebuilt unnecessarily.
        - bpftool: uncouple the copy of nlattr.h from libbpf target, to have it
          depend on the source header itself. By avoiding to reinstall this
          header every time, we avoid unnecessary builds of bpftool.
        - samples/bpf: Add a new commit to remove the FORCE dependency for
          libbpf, and replace it with a "$(wildcard ...)" on the .c/.h files in
          libbpf's directory. This is to avoid always recompiling libbpf/bpftool.
        - Adjust prefixes in commit subjects.
      
      v3:
        - Remove order-only dependencies on $(LIBBPF_INCLUDE) (or equivalent)
          directories, given that they are created by libbpf's Makefile.
        - Add libbpf as a dependency for bpftool/resolve_btfids/runqslower when
          they are supposed to reuse a libbpf compiled previously. This is to
          avoid having several libbpf versions being compiled simultaneously in
          the same directory with parallel builds. Even if this didn't show up
          during tests, let's remain on the safe side.
        - kernel/bpf/preload/Makefile: Rename libbpf-hdrs (dash) dependency as
          libbpf_hdrs.
        - samples/bpf/.gitignore: Add bpftool/
        - samples/bpf/Makefile: Change "/bin/rm -rf" to "$(RM) -r".
        - samples/bpf/Makefile: Add missing slashes for $(LIBBPF_OUTPUT) and
          $(LIBBPF_DESTDIR) when buildling bpftool
        - samples/bpf/Makefile: Add a dependency to libbpf's headers for
          $(TRACE_HELPERS).
        - bpftool's Makefile: Use $(LIBBPF) instead of equivalent (but longer)
          $(LIBBPF_OUTPUT)libbpf.a
        - BPF iterators' Makefile: build bpftool in .output/bpftool (instead of
          .output/), add and clean up variables.
        - runqslower's Makefile: Add an explicit dependency on libbpf's headers
          to several objects. The dependency is not required (libbpf should have
          been compiled and so the headers exported through other dependencies
          for those targets), but they better mark the logical dependency and
          should help if exporting the headers changed in the future.
        - New commit to add an "install-bin" target to bpftool, to avoid
          installing bash completion when buildling BPF iterators and selftests.
      
      v2: Declare an additional dependency on libbpf's headers for
          iterators/iterators.o in kernel/preload/Makefile to make sure that
          these headers are exported before we compile the object file (and not
          just before we link it).
      ====================
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      1c8dab7d
    • Quentin Monnet's avatar
      bpftool: Add install-bin target to install binary only · d7db0a4e
      Quentin Monnet authored
      With "make install", bpftool installs its binary and its bash completion
      file. Usually, this is what we want. But a few components in the kernel
      repository (namely, BPF iterators and selftests) also install bpftool
      locally before using it. In such a case, bash completion is not
      necessary and is just a useless build artifact.
      
      Let's add an "install-bin" target to bpftool, to offer a way to install
      the binary only.
      Signed-off-by: default avatarQuentin Monnet <quentin@isovalent.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20211007194438.34443-13-quentin@isovalent.com
      d7db0a4e
    • Quentin Monnet's avatar
      selftests/bpf: Better clean up for runqslower in test_bpftool_build.sh · 87ee33bf
      Quentin Monnet authored
      The script test_bpftool_build.sh attempts to build bpftool in the
      various supported ways, to make sure nothing breaks.
      
      One of those ways is to run "make tools/bpf" from the root of the kernel
      repository. This command builds bpftool, along with the other tools
      under tools/bpf, and runqslower in particular. After running the
      command and upon a successful bpftool build, the script attempts to
      cleanup the generated objects. However, after building with this target
      and in the case of runqslower, the files are not cleaned up as expected.
      
      This is because the "tools/bpf" target sets $(OUTPUT) to
      .../tools/bpf/runqslower/ when building the tool, causing the object
      files to be placed directly under the runqslower directory. But when
      running "cd tools/bpf; make clean", the value for $(OUTPUT) is set to
      ".output" (relative to the runqslower directory) by runqslower's
      Makefile, and this is where the Makefile looks for files to clean up.
      
      We cannot easily fix in the root Makefile (where "tools/bpf" is defined)
      or in tools/scripts/Makefile.include (setting $(OUTPUT)), where changing
      the way the output variables are passed would likely have consequences
      elsewhere. We could change runqslower's Makefile to build in the
      repository instead of in a dedicated ".output/", but doing so just to
      accommodate a test script doesn't sound great. Instead, let's just make
      sure that we clean up runqslower properly by adding the correct command
      to the script.
      
      This will attempt to clean runqslower twice: the first try with command
      "cd tools/bpf; make clean" will search for tools/bpf/runqslower/.output
      and fail to clean it (but will still clean the other tools, in
      particular bpftool), the second one (added in this commit) sets the
      $(OUTPUT) variable like for building with the "tool/bpf" target and
      should succeed.
      Signed-off-by: default avatarQuentin Monnet <quentin@isovalent.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20211007194438.34443-12-quentin@isovalent.com
      87ee33bf
    • Quentin Monnet's avatar
      samples/bpf: Do not FORCE-recompile libbpf · a60d24e7
      Quentin Monnet authored
      In samples/bpf/Makefile, libbpf has a FORCE dependency that force it to
      be rebuilt. I read this as a way to keep the library up-to-date, given
      that we do not have, in samples/bpf, a list of the source files for
      libbpf itself. However, a better approach would be to use the
      "$(wildcard ...)" function from make, and to have libbpf depend on all
      the .c and .h files in its directory. This is what samples/bpf/Makefile
      does for bpftool, and also what the BPF selftests' Makefile does for
      libbpf.
      
      Let's update the Makefile to avoid rebuilding libbpf all the time (and
      bpftool on top of it).
      Signed-off-by: default avatarQuentin Monnet <quentin@isovalent.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20211007194438.34443-11-quentin@isovalent.com
      a60d24e7
    • Quentin Monnet's avatar
      samples/bpf: Install libbpf headers when building · 3f7a3318
      Quentin Monnet authored
      API headers from libbpf should not be accessed directly from the source
      directory. Instead, they should be exported with "make install_headers".
      Make sure that samples/bpf/Makefile installs the headers properly when
      building.
      
      The object compiled from and exported by libbpf are now placed into a
      subdirectory of sample/bpf/ instead of remaining in tools/lib/bpf/. We
      attempt to remove this directory on "make clean". However, the "clean"
      target re-enters the samples/bpf/ directory from the root of the
      repository ("$(MAKE) -C ../../ M=$(CURDIR) clean"), in such a way that
      $(srctree) and $(src) are not defined, making it impossible to use
      $(LIBBPF_OUTPUT) and $(LIBBPF_DESTDIR) in the recipe. So we only attempt
      to clean $(CURDIR)/libbpf, which is the default value.
      
      Add a dependency on libbpf's headers for the $(TRACE_HELPERS).
      
      We also change the output directory for bpftool, to place the generated
      objects under samples/bpf/bpftool/ instead of building in bpftool's
      directory directly. Doing so, we make sure bpftool reuses the libbpf
      library previously compiled and installed.
      Signed-off-by: default avatarQuentin Monnet <quentin@isovalent.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20211007194438.34443-10-quentin@isovalent.com
      3f7a3318
    • Quentin Monnet's avatar
      samples/bpf: Update .gitignore · 62fde1c8
      Quentin Monnet authored
      Update samples/bpf/.gitignore to ignore files generated when building
      the samples. Add:
      
        - vmlinux.h
        - the generated skeleton files (*.skel.h)
        - the samples/bpf/libbpf/ and .../bpftool/ directories, in preparation
          of a future commit which introduces a local output directory for
          building libbpf and bpftool.
      Signed-off-by: default avatarQuentin Monnet <quentin@isovalent.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20211007194438.34443-9-quentin@isovalent.com
      62fde1c8
    • Quentin Monnet's avatar
      bpf: iterators: Install libbpf headers when building · 7bf731dc
      Quentin Monnet authored
      API headers from libbpf should not be accessed directly from the
      library's source directory. Instead, they should be exported with "make
      install_headers". Let's make sure that bpf/preload/iterators/Makefile
      installs the headers properly when building.
      Signed-off-by: default avatarQuentin Monnet <quentin@isovalent.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20211007194438.34443-8-quentin@isovalent.com
      7bf731dc
    • Quentin Monnet's avatar
      bpf: preload: Install libbpf headers when building · bf607917
      Quentin Monnet authored
      API headers from libbpf should not be accessed directly from the
      library's source directory. Instead, they should be exported with "make
      install_headers". Let's make sure that bpf/preload/Makefile installs the
      headers properly when building.
      
      Note that we declare an additional dependency for iterators/iterators.o:
      having $(LIBBPF_A) as a dependency to "$(obj)/bpf_preload_umd" is not
      sufficient, as it makes it required only at the linking step. But we
      need libbpf to be compiled, and in particular its headers to be
      exported, before we attempt to compile iterators.o. The issue would not
      occur before this commit, because libbpf's headers were not exported and
      were always available under tools/lib/bpf.
      Signed-off-by: default avatarQuentin Monnet <quentin@isovalent.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20211007194438.34443-7-quentin@isovalent.com
      bf607917
    • Quentin Monnet's avatar
      tools/runqslower: Install libbpf headers when building · be79505c
      Quentin Monnet authored
      API headers from libbpf should not be accessed directly from the
      library's source directory. Instead, they should be exported with "make
      install_headers". Let's make sure that runqslower installs the
      headers properly when building.
      
      We use a libbpf_hdrs target to mark the logical dependency on libbpf's
      headers export for a number of object files, even though the headers
      should have been exported at this time (since bpftool needs them, and is
      required to generate the skeleton or the vmlinux.h).
      
      When descending from a parent Makefile, the specific output directories
      for building the library and exporting the headers are configurable with
      BPFOBJ_OUTPUT and BPF_DESTDIR, respectively. This is in addition to
      OUTPUT, on top of which those variables are constructed by default.
      
      Also adjust the Makefile for the BPF selftests. We pass a number of
      variables to the "make" invocation, because we want to point runqslower
      to the (target) libbpf shared with other tools, instead of building its
      own version. In addition, runqslower relies on (target) bpftool, and we
      also want to pass the proper variables to its Makefile so that bpftool
      itself reuses the same libbpf.
      Signed-off-by: default avatarQuentin Monnet <quentin@isovalent.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20211007194438.34443-6-quentin@isovalent.com
      be79505c