New command line options have been added to tools/trace.py to support the 
new BUILD_ID stackmap. List of symbol files can be added to the script to 
resolve symbols from build id as reported by the kernel in the stack trace
Updated man page and added an example usage

profile.py: return kernel annotations for folded stacks

Recently kernel libbpf was imported in bcc. It makes some of the
libbcc code redundant. This patch modifies libbcc functions to use
libbpf APIs.

 ddos_detector.py to monitor DDOS attacks

Address issue 2154.

When a struct S is used as key to a BPF_HISTOGRAM, it is assumed that the second
member of S holds the slot. But when S is converted to python from bpf C,
a padding may be inserted as a second member. This breaks print_log2_hist().

    root@debian:~/bcc/tools# ./softirqs.py -d
    Tracing soft irq event time... Hit Ctrl-C to end.
    ^C
    Traceback (most recent call last):
      File "./softirqs.py", line 144, in <module>
        dist.print_log2_hist(label, "softirq", section_print_fn=vec_to_name)
      File "/usr/local/lib/python2.7/dist-packages/bcc/table.py", line 326, in print_log2_hist
        vals[slot] = v.value
    TypeError: list indices must be integers, not str

Fix it by skipping the possible padding. Future work would be fixing/working
around in the library where the padding is introduced.

use kernel libbpf in bcc

highlighting the <80 chars, and also adding a note about tool names.

This patch demonstrated how to use libbpf function calls.
Specially, the bcc_map_create is changed to use libbpf
functions bpf_create_map_name and bpf_create_map.
Signed-off-by: Yonghong Song <yhs@fb.com>

The libbpf/src/*.c files are included in the build system,
so those functions will be available for bcc internals to use them.

There are two name conflicts, bpf_create_map and bpf_prog_load,
between src/cc/libbpf.c and src/cc/libbpf/src/{bpf.c,libbpf.c}.
To keep src/cc/libbpf intact, the following renaming happened
in bcc repo:
  bpf_create_map  =>  bcc_create_map
  bpf_prog_load   =>  bcc_prog_load
Signed-off-by: Yonghong Song <yhs@fb.com>

The uapi header
  src/cc/libbpf/include/uapi/linux/bpf.h
is used instead of the current way
  src/cc/compat/linux/bpf.h
Signed-off-by: Yonghong Song <yhs@fb.com>

If the target directory src/cc/libbpf/ does not exist,
the top level CMakeLists.txt is changed to fetch
submodules with the following command
  git submodule update --init --recursive

The dev/rpm build scripts are also changed
to do git submodule update.
Signed-off-by: Yonghong Song <yhs@fb.com>

The kernel libbpf library
  https://github.com/torvalds/linux/tree/master/tools/lib/bpf
provides common functionality for handling bpf programs, maps,
ELF parsing, probing, BTF manipulation, etc. The library has
been recommended to the community as a common library for
bpf/btf handling. A separate repository
  https://github.com/libbpf/libbpf
has been created for this purpose.

This patch brings in libbpf to bcc. The following command
is executed to crate a libbpf submodule at src/cc/libbpf:
  git submodule add https://github.com/libbpf/libbpf.git src/cc/libbpf
Signed-off-by: Yonghong Song <yhs@fb.com>

This fixes the format string used to parse the major and minor
device fields in /proc/self/maps. These fields have hexadecimal
values and hence cannot be parsed as unsigned integers.

Fixes: 51480d05 ("implement free_bcc_memory() API (#2097)")
Reported-by: Nageswara R Sastry rnsastry@linux.vnet.ibm.com
Signed-off-by: Sandipan Das <sandipan@linux.ibm.com>

Across the repo (see libbpf.h, api/BPF.h) we always use the custom header search include, to make it easier to work with different build environments.

Some users uses `BPFModule` directly instead of C++ / Python API, and
they would like to have similar interface of free BCC `.text` memory

Unify shebangs under `#!/usr/bin/python`

* Mark unused parameters as unused

In Android, we are building libbpf with -Wunused-parameter, mark the
parameters in bpf_detach_tracepoint to prevent errors.

Change-Id: I2d0011746af80898e55d456b973a95330ce6be71
Signed-off-by: Joel Fernandes <joel@joelfernandes.org>

* Avoid void pointer arithmetic

In Android, we build libbpf with -Wpointer-arith, this causes warnings
as below. Avoid void pointer arithmetic to prevent the warning.

external/bcc/src/cc/perf_reader.c:189:26: error: arithmetic on a pointer
to void is a GNU extension [-Werror,-Wpointer-arith] memcpy(reader->buf
+ len, base, e->size - len);                               ~~~~~~~~~~~ ^

Change-Id: If06535459473c78799b38119786a91e74a208895
Signed-off-by: Joel Fernandes <joel@joelfernandes.org>

* Cast correctly for unsigned long format specifier

In Android, -Wformat gets passed to the compiler causing a warning. Fix
it by casting.

external/bcc/src/cc/libbpf.c:972:58: error: format specifies type
'unsigned long' but the argument has type 'uint64_t' (aka 'unsigned lo
ng long') [-Werror,-Wformat]

Change-Id: I5e70eeff983f20a0b921e81aee7ddbee6d7de2b3
Signed-off-by: Joel Fernandes <joel@joelfernandes.org>

A separate build_id stackmap can be created with the help of new macro BPF_STACK_TRACE_BUILDID. The kernel BPF reports stacktrace in the 
structure bpf_stack_build_id. Changes have been made to BPF modules to
support resolving symbols mentioned in the above format.
An example tool is also available in examples/tracing/stack_buildid_example.py. 
Both python and C++ test cases are added.

On x64 from kernel v4.17 onwards, a indirect table is used to
rewrite syscall parameters in trace.py. However, it only works
for arg1. This patch fixes it for arg2-arg6 too.

Fixes: 2da34267 ("generate indirect parameter assignment if arch uses syscall wrapper (#1816)")
Signed-off-by: Xiaozhou Liu <liuxiaozhou@bytedance.com>

debian changelog for v0.8.0 tag

Misc fixes on Python tools

syscall.py: Fixes python3 related error

With out the patch:
Traceback (most recent call last):$
  File "/root/bcc/src/python/bcc/syscall.py", line 381, in <module>$
    out = out.split('\n',1)[1]$
TypeError: a bytes-like object is required, not 'str'$
$
During handling of the above exception, another exception occurred:$
$
Traceback (most recent call last):$
  File "/root/bcc/tests/python/test_stat1.py", line 10, in <module>$
    from bcc import BPF$
  File "/root/bcc/src/python/bcc/__init__.py", line 30, in <module>$
    from .syscall import syscall_name$
  File "/root/bcc/src/python/bcc/syscall.py", line 387, in <module>$
    raise Exception("ausyscall: command not found")$
Exception: ausyscall: command not found$

This is because variable 'out' is a byte object type and while split,
code is passing 'str' type.

Tested this on python3 and python2
Signed-off-by: Nageswara R Sastry <rnsastry@linux.vnet.ibm.com>

Handle keyboard interrupt to avoid backtrace printed out when user
presses Ctrl-C. Other tools have been fixed recently. Let's fix these
remaining tools.

Fix issue #2119.

Kernel 5.0 added more usages of asm goto and llvm
does not support asm goto yet. This resulted in
compilation error for virtually any bcc scripts.

The workaround here is to redefine asm_volatile_goto to
  asm volatile("invalid use of asm_volatile_goto")
which can pass clang.
If bpf program does not use asm_volatile_goto,
nothing bad will happen. The functions using
asm_volatile_goto will be thrown away.
If bpf program accidentally uses asm_volatile_goto,
a compilation error like below will be printed out:
  <inline asm>:1:2: error: invalid register/token name
          invalid use of asm_volatile_goto
          ^
  LLVM ERROR: Error parsing inline asm
Signed-off-by: Yonghong Song <yhs@fb.com>

add new Ubuntu bpfcc-tools package name in INSTALL.md

add missing PT_REGS_RET definition for x86

Fix BPFModule error when RW Engine is disabled 

fix several printb usage with python3

The following three tools are recently changed to use
printb in order to flush out the result.
  opensnoop.py, tcpaccept.py, tcpconnect.py

With python3, however, these tools have errors like below:
  TypeError: %b requires a bytes-like object, or an object that implements __bytes__, not 'str'
  Traceback (most recent call last):
    File "_ctypes/callbacks.c", line 234, in 'calling callback function'
    File "/usr/lib/python3.6/site-packages/bcc/table.py", line 572, in raw_cb_
      callback(cpu, data, size)
    File "../../tools/opensnoop.py", line 248, in print_event
      printb(b'%s' % event.fname.decode('utf-8', 'replace'))

This patch fixed printb related issues for these three tools
for python3. The python2 still works with the fix.
Signed-off-by: Yonghong Song <yhs@fb.com>

The bpf program generated with memset() call in this tool contains
BPF_JMP|BPF_CALL instruction with imm=-1. The verifier rejects the
program. Let's use llvm built-in function __builtin_memset

On fc29, compiling bcc from source, the following
compiler warnings are seen:
  /home/yhs/work/bcc/src/cc/bcc_elf.c: In function ‘bcc_free_memory_with_file’:
  /home/yhs/work/bcc/src/cc/bcc_elf.c:802:36: warning: unused variable ‘_fini’ [-Wunused-variable]
       extern unsigned long _start, _fini;
                                    ^~~~~
/home/yhs/work/bcc/src/cc/bcc_elf.c:802:28: warning: unused variable ‘_start’ [-Wunused-variable]
       extern unsigned long _start, _fini;
                            ^~~~~~

These unused externs are accidentally introduced by
Commit 51480d05 ("implement free_bcc_memory() API").
This patch removed them.
Signed-off-by: Yonghong Song <yhs@fb.com>

* python: remove MANIFEST

* cmake/python: fix build race condition

When ARCH is not defined, bcc get the architecture from uname. It then
modifies it to get the name of arch directory in linux source.

When ARCH is defined however, it just copy it as is, without the
translation to the arch directory. If for instance ARCH is set to
x86_64, it tries to look into the include directory
build/arch/x86_64/, which doesn't exist.

It fixes the following issue:
$ echo $ARCH
x86_64
$ /usr/share/bcc/tools/bashreadline
In file included from <built-in>:2:
In file included from /virtual/include/bcc/bpf.h:12:
In file included from /lib/modules/4.18.0-49.el8.x86_64/build/include/linux/types.h:6:
/lib/modules/4.18.0-49.el8.x86_64/build/include/uapi/linux/types.h:5:10: fatal error: 'asm/types.h' file
      not found
         ^~~~~~~~~~~~~
1 error generated.
Traceback (most recent call last):
  File "/usr/share/bcc/tools/bashreadline", line 51, in <module>
    b = BPF(text=bpf_text)
  File "/usr/lib/python3.6/site-packages/bcc/__init__.py", line 318, in __init__
    raise Exception("Failed to compile BPF text")
Exception: Failed to compile BPF text

support uid option in tcpconnect.

bpf_trace_printk() does not have `fmt_size` as the second parameter in BCC.