1. 27 Feb, 2019 2 commits
    • Alexei Starovoitov's avatar
      bpf: expose program stats via bpf_prog_info · 5f8f8b93
      Alexei Starovoitov authored
      Return bpf program run_time_ns and run_cnt via bpf_prog_info
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Acked-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      5f8f8b93
    • Alexei Starovoitov's avatar
      bpf: enable program stats · 492ecee8
      Alexei Starovoitov authored
      JITed BPF programs are indistinguishable from kernel functions, but unlike
      kernel code BPF code can be changed often.
      Typical approach of "perf record" + "perf report" profiling and tuning of
      kernel code works just as well for BPF programs, but kernel code doesn't
      need to be monitored whereas BPF programs do.
      Users load and run large amount of BPF programs.
      These BPF stats allow tools monitor the usage of BPF on the server.
      The monitoring tools will turn sysctl kernel.bpf_stats_enabled
      on and off for few seconds to sample average cost of the programs.
      Aggregated data over hours and days will provide an insight into cost of BPF
      and alarms can trigger in case given program suddenly gets more expensive.
      
      The cost of two sched_clock() per program invocation adds ~20 nsec.
      Fast BPF progs (like selftests/bpf/progs/test_pkt_access.c) will slow down
      from ~10 nsec to ~30 nsec.
      static_key minimizes the cost of the stats collection.
      There is no measurable difference before/after this patch
      with kernel.bpf_stats_enabled=0
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      492ecee8
  2. 25 Feb, 2019 7 commits
    • Daniel Borkmann's avatar
      Merge branch 'bpf-libbpf-af-xdp' · 143bdc2e
      Daniel Borkmann authored
      Magnus Karlsson says:
      
      ====================
      This patch proposes to add AF_XDP support to libbpf. The main reason
      for this is to facilitate writing applications that use AF_XDP by
      offering higher-level APIs that hide many of the details of the AF_XDP
      uapi. This is in the same vein as libbpf facilitates XDP adoption by
      offering easy-to-use higher level interfaces of XDP
      functionality. Hopefully this will facilitate adoption of AF_XDP, make
      applications using it simpler and smaller, and finally also make it
      possible for applications to benefit from optimizations in the AF_XDP
      user space access code. Previously, people just copied and pasted the
      code from the sample application into their application, which is not
      desirable.
      
      The proposed interface is composed of two parts:
      
      * Low-level access interface to the four rings and the packet
      * High-level control plane interface for creating and setting up umems
        and AF_XDP sockets. This interface also loads a simple XDP program
        that routes all traffic on a queue up to the AF_XDP socket.
      
      The sample program has been updated to use this new interface and in
      that process it lost roughly 300 lines of code. I cannot detect any
      performance degradations due to the use of this library instead of the
      previous functions that were inlined in the sample application. But I
      did measure this on a slower machine and not the Broadwell that we
      normally use.
      
      The rings are now called xsk_ring and when a producer operates on
      it. It is xsk_ring_prod and for a consumer it is xsk_ring_cons. This
      way we can get some compile time error checking that the rings are
      used correctly.
      
      Comments and contenplations:
      
      * The current behaviour is that the library loads an XDP program (if
        requested to do so) but the clean up of this program is left to the
        application. It would be possible to implement this cleanup in the
        library, but it would require state to be kept on netdev level,
        which there is none at the moment, and the synchronization of this
        between processes. All this adding complexity. But when we get an
        XDP program per queue id, then it becomes trivial to also remove the
        XDP program when the application exits. This proposal from Jesper,
        Björn and others will also improve the performance of libbpf, since
        most of the XDP program code can be removed when that feature is
        supported.
      
      * In a future release, I am planning on adding a higher level data
        plane interface too. This will be based around recvmsg and sendmsg
        with the use of struct iovec for batching, without the user having
        to know anything about the underlying four rings of an AF_XDP
        socket. There will be one semantic difference though from the
        standard recvmsg and that is that the kernel will fill in the iovecs
        instead of the application. But the rest should be the same as the
        libc versions so that application writers feel at home.
      
      Patch 1: adds AF_XDP support in libbpf
      Patch 2: updates the xdpsock sample application to use the libbpf functions
      Patch 3: Documentation update to help first time users
      
      Changes v5 to v6:
        * Fixed prog_fd bug found by Xiaolong Ye. Thanks!
      Changes v4 to v5:
        * Added a FAQ to the documentation
        * Removed xsk_umem__get_data and renamed xsk_umem__get_dat_raw to
          xsk_umem__get_data
        * Replaced the netlink code with bpf_get_link_xdp_id()
        * Dynamic allocation of the map sizes. They are now sized after
          the max number of queueus on the netdev in question.
      Changes v3 to v4:
        * Dropped the pr_*() patch in favor of Yonghong Song's patch set
        * Addressed the review comments of Daniel Borkmann, mainly leaking
          of file descriptors at clean up and making the data plane APIs
          all static inline (with the exception of xsk_umem__get_data that
          uses an internal structure I do not want to expose).
        * Fixed the netlink callback as suggested by Maciej Fijalkowski.
        * Removed an unecessary include in the sample program as spotted by
          Ilia Fillipov.
      Changes v2 to v3:
        * Added automatic loading of a simple XDP program that routes all
          traffic on a queue up to the AF_XDP socket. This program loading
          can be disabled.
        * Updated function names to be consistent with the libbpf naming
          convention
        * Moved all code to xsk.[ch]
        * Removed all the XDP program loading code from the sample since
          this is now done by libbpf
        * The initialization functions now return a handle as suggested by
          Alexei
        * const statements added in the API where applicable.
      Changes v1 to v2:
        * Fixed cleanup of library state on error.
        * Moved API to initial version
        * Prefixed all public functions by xsk__ instead of xsk_
        * Added comment about changed default ring sizes, batch size and umem
          size in the sample application commit message
        * The library now only creates an Rx or Tx ring if the respective
          parameter is != NULL
      ====================
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      143bdc2e
    • Magnus Karlsson's avatar
      xsk: add FAQ to facilitate for first time users · 0f4a9b7d
      Magnus Karlsson authored
      Added an FAQ section in Documentation/networking/af_xdp.rst to help
      first time users with common problems. As problems are getting
      identified, entries will be added to the FAQ.
      Signed-off-by: default avatarMagnus Karlsson <magnus.karlsson@intel.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      0f4a9b7d
    • Magnus Karlsson's avatar
      samples/bpf: convert xdpsock to use libbpf for AF_XDP access · 248c7f9c
      Magnus Karlsson authored
      This commit converts the xdpsock sample application to use the AF_XDP
      functions present in libbpf. This cuts down the size of it by nearly
      300 lines of code.
      
      The default ring sizes plus the batch size has been increased and the
      size of the umem area has decreased. This so that the sample application
      will provide higher throughput. Note also that the shared umem code
      has been removed from the sample as this is not supported by libbpf
      at this point in time.
      Tested-by: default avatarBjörn Töpel <bjorn.topel@intel.com>
      Signed-off-by: default avatarMagnus Karlsson <magnus.karlsson@intel.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      248c7f9c
    • Magnus Karlsson's avatar
      libbpf: add support for using AF_XDP sockets · 1cad0788
      Magnus Karlsson authored
      This commit adds AF_XDP support to libbpf. The main reason for this is
      to facilitate writing applications that use AF_XDP by offering
      higher-level APIs that hide many of the details of the AF_XDP
      uapi. This is in the same vein as libbpf facilitates XDP adoption by
      offering easy-to-use higher level interfaces of XDP
      functionality. Hopefully this will facilitate adoption of AF_XDP, make
      applications using it simpler and smaller, and finally also make it
      possible for applications to benefit from optimizations in the AF_XDP
      user space access code. Previously, people just copied and pasted the
      code from the sample application into their application, which is not
      desirable.
      
      The interface is composed of two parts:
      
      * Low-level access interface to the four rings and the packet
      * High-level control plane interface for creating and setting
        up umems and af_xdp sockets as well as a simple XDP program.
      Tested-by: default avatarBjörn Töpel <bjorn.topel@intel.com>
      Signed-off-by: default avatarMagnus Karlsson <magnus.karlsson@intel.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      1cad0788
    • Stanislav Fomichev's avatar
      selftests/bpf: make sure signal interrupts BPF_PROG_TEST_RUN · 740f8a65
      Stanislav Fomichev authored
      Simple test that I used to reproduce the issue in the previous commit:
      Do BPF_PROG_TEST_RUN with max iterations, each program is 4096 simple
      move instructions. File alarm in 0.1 second and check that
      bpf_prog_test_run is interrupted (i.e. test doesn't hang).
      
      Note: reposting this for bpf-next to avoid linux-next conflict. In this
      version I test both BPF_PROG_TYPE_SOCKET_FILTER (which uses generic
      bpf_test_run implementation) and BPF_PROG_TYPE_FLOW_DISSECTOR (which has
      it own loop with preempt handling in bpf_prog_test_run_flow_dissector).
      Signed-off-by: default avatarStanislav Fomichev <sdf@google.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      740f8a65
    • Stanislav Fomichev's avatar
      bpf/test_run: fix unkillable BPF_PROG_TEST_RUN for flow dissector · a439184d
      Stanislav Fomichev authored
      Syzbot found out that running BPF_PROG_TEST_RUN with repeat=0xffffffff
      makes process unkillable. The problem is that when CONFIG_PREEMPT is
      enabled, we never see need_resched() return true. This is due to the
      fact that preempt_enable() (which we do in bpf_test_run_one on each
      iteration) now handles resched if it's needed.
      
      Let's disable preemption for the whole run, not per test. In this case
      we can properly see whether resched is needed.
      Let's also properly return -EINTR to the userspace in case of a signal
      interrupt.
      
      This is a follow up for a recently fixed issue in bpf_test_run, see
      commit df1a2cb7 ("bpf/test_run: fix unkillable
      BPF_PROG_TEST_RUN").
      Reported-by: default avatarsyzbot <syzkaller@googlegroups.com>
      Signed-off-by: default avatarStanislav Fomichev <sdf@google.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      a439184d
    • Anders Roxell's avatar
      bpf: test_bpf: turn off preemption in function __run_once · fd92d664
      Anders Roxell authored
      When running BPF test suite the following splat occurs:
      
      [  415.930950] test_bpf: #0 TAX jited:0
      [  415.931067] BUG: assuming atomic context at lib/test_bpf.c:6674
      [  415.946169] in_atomic(): 0, irqs_disabled(): 0, pid: 11556, name: modprobe
      [  415.953176] INFO: lockdep is turned off.
      [  415.957207] CPU: 1 PID: 11556 Comm: modprobe Tainted: G        W         5.0.0-rc7-next-20190220 #1
      [  415.966328] Hardware name: HiKey Development Board (DT)
      [  415.971592] Call trace:
      [  415.974069]  dump_backtrace+0x0/0x160
      [  415.977761]  show_stack+0x24/0x30
      [  415.981104]  dump_stack+0xc8/0x114
      [  415.984534]  __cant_sleep+0xf0/0x108
      [  415.988145]  test_bpf_init+0x5e0/0x1000 [test_bpf]
      [  415.992971]  do_one_initcall+0x90/0x428
      [  415.996837]  do_init_module+0x60/0x1e4
      [  416.000614]  load_module+0x1de0/0x1f50
      [  416.004391]  __se_sys_finit_module+0xc8/0xe0
      [  416.008691]  __arm64_sys_finit_module+0x24/0x30
      [  416.013255]  el0_svc_common+0x78/0x130
      [  416.017031]  el0_svc_handler+0x38/0x78
      [  416.020806]  el0_svc+0x8/0xc
      
      Rework so that preemption is disabled when we loop over function
      'BPF_PROG_RUN(...)'.
      
      Fixes: 568f1967 ("bpf: check that BPF programs run with preemption disabled")
      Suggested-by: default avatarArnd Bergmann <arnd@arndb.de>
      Signed-off-by: default avatarAnders Roxell <anders.roxell@linaro.org>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      fd92d664
  3. 22 Feb, 2019 1 commit
  4. 21 Feb, 2019 1 commit
  5. 19 Feb, 2019 12 commits
  6. 18 Feb, 2019 3 commits
  7. 17 Feb, 2019 14 commits