1. 09 Mar, 2023 6 commits
  2. 08 Mar, 2023 24 commits
  3. 07 Mar, 2023 10 commits
    • Andrii Nakryiko's avatar
      Merge branch 'libbpf: usdt arm arg parsing support' · d1d51a62
      Andrii Nakryiko authored
      Puranjay Mohan says:
      
      ====================
      
      This series add the support of the ARM architecture to libbpf USDT. This
      involves implementing the parse_usdt_arg() function for ARM.
      
      It was seen that the last part of parse_usdt_arg() is repeated for all architectures,
      so, the first patch in this series refactors these functions and moved the post
      processing to parse_usdt_spec()
      
      Changes in V2[1] to V3:
      
      - Use a tabular approach to find register offsets.
      - Add the patch for refactoring parse_usdt_arg()
      ====================
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      d1d51a62
    • Puranjay Mohan's avatar
      libbpf: USDT arm arg parsing support · 720d93b6
      Puranjay Mohan authored
      Parsing of USDT arguments is architecture-specific; on arm it is
      relatively easy since registers used are r[0-10], fp, ip, sp, lr,
      pc. Format is slightly different compared to aarch64; forms are
      
      - "size @ [ reg, #offset ]" for dereferences, for example
        "-8 @ [ sp, #76 ]" ; " -4 @ [ sp ]"
      - "size @ reg" for register values; for example
        "-4@r0"
      - "size @ #value" for raw values; for example
        "-8@#1"
      
      Add support for parsing USDT arguments for ARM architecture.
      
      To test the above changes QEMU's virt[1] board with cortex-a15
      CPU was used. libbpf-bootstrap's usdt example[2] was modified to attach
      to a test program with DTRACE_PROBE1/2/3/4... probes to test different
      combinations.
      
      [1] https://www.qemu.org/docs/master/system/arm/virt.html
      [2] https://github.com/libbpf/libbpf-bootstrap/blob/master/examples/c/usdt.bpf.cSigned-off-by: default avatarPuranjay Mohan <puranjay12@gmail.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20230307120440.25941-3-puranjay12@gmail.com
      720d93b6
    • Puranjay Mohan's avatar
      libbpf: Refactor parse_usdt_arg() to re-use code · 98e678e9
      Puranjay Mohan authored
      The parse_usdt_arg() function is defined differently for each
      architecture but the last part of the function is repeated
      verbatim for each architecture.
      
      Refactor parse_usdt_arg() to fill the arg_sz and then do the repeated
      post-processing in parse_usdt_spec().
      Signed-off-by: default avatarPuranjay Mohan <puranjay12@gmail.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20230307120440.25941-2-puranjay12@gmail.com
      98e678e9
    • Daniel Müller's avatar
      libbpf: Fix theoretical u32 underflow in find_cd() function · 3ecde218
      Daniel Müller authored
      Coverity reported a potential underflow of the offset variable used in
      the find_cd() function. Switch to using a signed 64 bit integer for the
      representation of offset to make sure we can never underflow.
      
      Fixes: 1eebcb60 ("libbpf: Implement basic zip archive parsing support")
      Signed-off-by: default avatarDaniel Müller <deso@posteo.net>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Link: https://lore.kernel.org/bpf/20230307215504.837321-1-deso@posteo.net
      3ecde218
    • Alexei Starovoitov's avatar
      Merge branch 'bpf: bpf memory usage' · a73dc912
      Alexei Starovoitov authored
      Yafang Shao says:
      
      ====================
      
      Currently we can't get bpf memory usage reliably either from memcg or
      from bpftool.
      
      In memcg, there's not a 'bpf' item in memory.stat, but only 'kernel',
      'sock', 'vmalloc' and 'percpu' which may related to bpf memory. With
      these items we still can't get the bpf memory usage, because bpf memory
      usage may far less than the kmem in a memcg, for example, the dentry may
      consume lots of kmem.
      
      bpftool now shows the bpf memory footprint, which is difference with bpf
      memory usage. The difference can be quite great in some cases, for example,
      
      - non-preallocated bpf map
        The non-preallocated bpf map memory usage is dynamically changed. The
        allocated elements count can be from 0 to the max entries. But the
        memory footprint in bpftool only shows a fixed number.
      
      - bpf metadata consumes more memory than bpf element
        In some corner cases, the bpf metadata can consumes a lot more memory
        than bpf element consumes. For example, it can happen when the element
        size is quite small.
      
      - some maps don't have key, value or max_entries
        For example the key_size and value_size of ringbuf is 0, so its
        memlock is always 0.
      
      We need a way to show the bpf memory usage especially there will be more
      and more bpf programs running on the production environment and thus the
      bpf memory usage is not trivial.
      
      This patchset introduces a new map ops ->map_mem_usage to calculate the
      memory usage. Note that we don't intend to make the memory usage 100%
      accurate, while our goal is to make sure there is only a small difference
      between what bpftool reports and the real memory. That small difference
      can be ignored compared to the total usage.  That is enough to monitor
      the bpf memory usage. For example, the user can rely on this value to
      monitor the trend of bpf memory usage, compare the difference in bpf
      memory usage between different bpf program versions, figure out which
      maps consume large memory, and etc.
      
      This patchset implements the bpf memory usage for all maps, and yet there's
      still work to do. We don't want to introduce runtime overhead in the
      element update and delete path, but we have to do it for some
      non-preallocated maps,
      - devmap, xskmap
        When we update or delete an element, it will allocate or free memory.
        In order to track this dynamic memory, we have to track the count in
        element update and delete path.
      
      - cpumap
        The element size of each cpumap element is not determinated. If we
        want to track the usage, we have to count the size of all elements in
        the element update and delete path. So I just put it aside currently.
      
      - local_storage, bpf_local_storage
        When we attach or detach a cgroup, it will allocate or free memory. If
        we want to track the dynamic memory, we also need to do something in
        the update and delete path. So I just put it aside currently.
      
      - offload map
        The element update and delete of offload map is via the netdev dev_ops,
        in which it may dynamically allocate or free memory, but this dynamic
        memory isn't counted in offload map memory usage currently.
      
      The result of each map can be found in the individual patch.
      
      We may also need to track per-container bpf memory usage, that will be
      addressed by a different patchset.
      
      Changes:
      v3->v4: code improvement on ringbuf (Andrii)
              use READ_ONCE() to read lpm_trie (Tao)
              explain why we can't get bpf memory usage from memcg.
      v2->v3: check callback at map creation time and avoid warning (Alexei)
              fix build error under CONFIG_BPF=n (lkp@intel.com)
      v1->v2: calculate the memory usage within bpf (Alexei)
      - [v1] bpf, mm: bpf memory usage
        https://lwn.net/Articles/921991/
      - [RFC PATCH v2] mm, bpf: Add BPF into /proc/meminfo
        https://lwn.net/Articles/919848/
      - [RFC PATCH v1] mm, bpf: Add BPF into /proc/meminfo
        https://lwn.net/Articles/917647/
      - [RFC PATCH] bpf, mm: Add a new item bpf into memory.stat
        https://lore.kernel.org/bpf/20220921170002.29557-1-laoar.shao@gmail].com/
      ====================
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      a73dc912
    • Yafang Shao's avatar
      bpf: enforce all maps having memory usage callback · 6b4a6ea2
      Yafang Shao authored
      We have implemented memory usage callback for all maps, and we enforce
      any newly added map having a callback as well. We check this callback at
      map creation time. If it doesn't have the callback, we will return
      EINVAL.
      Signed-off-by: default avatarYafang Shao <laoar.shao@gmail.com>
      Link: https://lore.kernel.org/r/20230305124615.12358-19-laoar.shao@gmail.comSigned-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      6b4a6ea2
    • Yafang Shao's avatar
      bpf: offload map memory usage · 9629363c
      Yafang Shao authored
      A new helper is introduced to calculate offload map memory usage. But
      currently the memory dynamically allocated in netdev dev_ops, like
      nsim_map_update_elem, is not counted. Let's just put it aside now.
      Signed-off-by: default avatarYafang Shao <laoar.shao@gmail.com>
      Link: https://lore.kernel.org/r/20230305124615.12358-18-laoar.shao@gmail.comSigned-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      9629363c
    • Yafang Shao's avatar
      bpf, net: xskmap memory usage · b4fd0d67
      Yafang Shao authored
      A new helper is introduced to calculate xskmap memory usage.
      
      The xfsmap memory usage can be dynamically changed when we add or remove
      a xsk_map_node. Hence we need to track the count of xsk_map_node to get
      its memory usage.
      
      The result as follows,
      - before
      10: xskmap  name count_map  flags 0x0
              key 4B  value 4B  max_entries 65536  memlock 524288B
      
      - after
      10: xskmap  name count_map  flags 0x0 <<< no elements case
              key 4B  value 4B  max_entries 65536  memlock 524608B
      Signed-off-by: default avatarYafang Shao <laoar.shao@gmail.com>
      Link: https://lore.kernel.org/r/20230305124615.12358-17-laoar.shao@gmail.comSigned-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      b4fd0d67
    • Yafang Shao's avatar
      bpf, net: sock_map memory usage · 73d2c619
      Yafang Shao authored
      sockmap and sockhash don't have something in common in allocation, so let's
      introduce different helpers to calculate their memory usage.
      
      The reuslt as follows,
      
      - before
      28: sockmap  name count_map  flags 0x0
              key 4B  value 4B  max_entries 65536  memlock 524288B
      29: sockhash  name count_map  flags 0x0
              key 4B  value 4B  max_entries 65536  memlock 524288B
      
      - after
      28: sockmap  name count_map  flags 0x0
              key 4B  value 4B  max_entries 65536  memlock 524608B
      29: sockhash  name count_map  flags 0x0  <<<< no updated elements
              key 4B  value 4B  max_entries 65536  memlock 1048896B
      Signed-off-by: default avatarYafang Shao <laoar.shao@gmail.com>
      Link: https://lore.kernel.org/r/20230305124615.12358-16-laoar.shao@gmail.comSigned-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      73d2c619
    • Yafang Shao's avatar
      bpf, net: bpf_local_storage memory usage · 7490b7f1
      Yafang Shao authored
      A new helper is introduced into bpf_local_storage map to calculate the
      memory usage. This helper is also used by other maps like
      bpf_cgrp_storage, bpf_inode_storage, bpf_task_storage and etc.
      
      Note that currently the dynamically allocated storage elements are not
      counted in the usage, since it will take extra runtime overhead in the
      elements update or delete path. So let's put it aside now, and implement
      it in the future when someone really need it.
      Signed-off-by: default avatarYafang Shao <laoar.shao@gmail.com>
      Link: https://lore.kernel.org/r/20230305124615.12358-15-laoar.shao@gmail.comSigned-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      7490b7f1