1. 26 Jan, 2019 17 commits
    • Alexei Starovoitov's avatar
      Merge branch 'jmp32-insns' · ae575c8a
      Alexei Starovoitov authored
      Jiong Wang says:
      
      ====================
      v3 -> v4:
       - Fixed rebase issue. JMP32 checks were missing in two new functions:
          + kernel/bpf/verifier.c:insn_is_cond_jump
          + drivers/net/ethernet/netronome/nfp/bpf/main.h:is_mbpf_cond_jump
         (Daniel)
       - Further rebased on top of latest llvm-readelf change.
      
      v2 -> v3:
       - Added missed check on JMP32 inside bpf_jit_build_body. (Sandipan)
       - Wrap ?: statements in s390 port with brace. They are used by macros
         which doesn't guard the operand with brace.
       - Fixed the ',' issues test_verifier change.
       - Reorder two selftests patches to be near each other.
       - Rebased on top of latest bpf-next.
      
      v1 -> v2:
       - Updated encoding. Use reserved insn class 0x6 instead of packing with
         existing BPF_JMP. (Alexei)
       - Updated code comments in s390 port. (Martin)
       - Separate JIT function for jeq32_imm in NFP port. (Jakub)
       - Re-implemented auto-testing support. (Jakub)
       - Moved testcases to test_verifer.c, plus more unit tests. (Jakub)
       - Fixed JEQ/JNE range deduction. (Jakub)
       - Also supported JSET in this patch set.
       - Fixed/Improved range deduction for all the other operations. All C
         programs under bpf selftest passed verification now.
       - Improved min/max code implementation.
       - Fixed bpftool/disassembler.
      
      Current eBPF ISA has 32-bit sub-register and has defined a set of ALU32
      instructions.
      
      However, there is no JMP32 instructions, the consequence is code-gen for
      32-bit sub-registers is not efficient. For example, explicit sign-extension
      from 32-bit to 64-bit is needed for signed comparison.
      
      Adding JMP32 instruction therefore could complete eBPF ISA on 32-bit
      sub-register support. This also match those JMP32 instructions in most JIT
      backends, for example x64-64 and AArch64. These new eBPF JMP32 instructions
      could have one-to-one map on them.
      
      A few verifier ALU32 related bugs has been fixed recently, and JMP32
      introduced by this set further improves BPF sub-register ecosystem. Once
      this is landed, BPF programs using 32-bit sub-register ISA could get
      reasonably good support from verifier and JIT compilers. Users then could
      compare the runtime efficiency of one BPF program under both modes, and
      could use the one shown better from benchmark result.
      
      From benchmark results on some Cilium BPF programs, for 64-bit arches,
      after JMP32 introduced, programs compiled with -mattr=+alu32 (meaning
      enable sub-register usage) are smaller in code size and generally smaller
      in verifier processed insn number.
      
      Benchmark results
      ===
      Text size in bytes (generated by "size")
      ---
      LLVM code-gen option   default  alu32  alu32/jmp32  change Vs.  change Vs.
                                                          alu32       default
      bpf_lb-DLB_L3.o:       6456     6280   6160         -1.91%      -4.58%
      bpf_lb-DLB_L4.o:       7848     7664   7136         -6.89%      -9.07%
      bpf_lb-DUNKNOWN.o:     2680     2664   2568         -3.60%      -4.18%
      bpf_lxc.o:             104824   104744 97360        -7.05%      -7.12%
      bpf_netdev.o:          23456    23576  21632        -8.25%      -7.78%
      bpf_overlay.o:         16184    16304  14648        -10.16%     -9.49%
      
      Processed instruction number
      ---
      LLVM code-gen option   default  alu32  alu32/jmp32  change Vs.  change Vs.
                                                          alu32       default
      bpf_lb-DLB_L3.o:       1579     1281   1295         +1.09%      -17.99%
      bpf_lb-DLB_L4.o:       2045     1663   1556         -6.43%      -23.91%
      bpf_lb-DUNKNOWN.o:     606      513    501          -2.34%      -17.33%
      bpf_lxc.o:             85381    103218 94435        -8.51%      +10.60%
      bpf_netdev.o:          5246     5809   5200         -10.48%     -0.08%
      bpf_overlay.o:         2443     2705   2456         -9.02%      -0.53%
      
      It is even better for 32-bit arches like x32, arm32 and nfp etc, as now
      some conditional jump will become JMP32 which doesn't require code-gen for
      high 32-bit comparison.
      
      Encoding
      ===
      The new JMP32 instructions are using new BPF_JMP32 class which is using
      the reserved eBPF class number 0x6. And BPF_JA/CALL/EXIT only exist for
      BPF_JMP, they are reserved opcode for BPF_JMP32.
      
      LLVM support
      ===
      A couple of unit tests has been added and included in this set. Also LLVM
      code-gen for JMP32 has been added, so you could just compile any BPF C
      program with both -mcpu=probe and -mattr=+alu32 specified. If you are
      compiling on a machine with kernel patched by this set, LLVM will select
      the ISA automatically based on host probe results. Otherwise specify
      -mcpu=v3 and -mattr=+alu32 could also force use JMP32 ISA.
      
         LLVM support could be found at:
      
           https://github.com/Netronome/llvm/tree/jmp32-v2
      
         (clang driver also taught about the new "v3" processor, will send out
          merge request for both clang and llvm once kernel set landed.)
      
      JIT backends support
      ===
      A couple of JIT backends has been supported in this set except SPARC and
      MIPS. It shouldn't be a big issue for these two ports as LLVM default won't
      generate JMP32 insns, it will only generate them when host machine is
      probed to be with the support.
      
      Thanks.
      ====================
      Acked-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      ae575c8a
    • Jiong Wang's avatar
      selftests: bpf: makefile support sub-register code-gen test mode · 3ef84346
      Jiong Wang authored
      This patch enables testing some eBPF programs under sub-register
      compilation mode.
      
      Only enable this when there is BPF_JMP32 support on both LLVM and kernel.
      This is because only after BPF_JMP32 added, code-gen for complex program
      under sub-register mode will be clean enough to pass verification.
      
      This patch splits TEST_GEN_FILES into BPF_OBJ_FILES and
      BPF_OBJ_FILES_DUAL_COMPILE. The latter are those objects we would like to
      compile for both default and sub-register mode. They are also objects used
      by "test_progs".
      Reviewed-by: default avatarJakub Kicinski <jakub.kicinski@netronome.com>
      Signed-off-by: default avatarJiong Wang <jiong.wang@netronome.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      3ef84346
    • Jiong Wang's avatar
      selftests: bpf: functional and min/max reasoning unit tests for JMP32 · 6ea848b5
      Jiong Wang authored
      This patch adds unit tests for new JMP32 instructions.
      
      This patch also added the new BPF_JMP32_REG and BPF_JMP32_IMM macros to
      samples/bpf/bpf_insn.h so that JMP32 insn builders are available to tests
      under 'samples' directory.
      Reviewed-by: default avatarJakub Kicinski <jakub.kicinski@netronome.com>
      Signed-off-by: default avatarJiong Wang <jiong.wang@netronome.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      6ea848b5
    • Jiong Wang's avatar
      nfp: bpf: implement jitting of JMP32 · 46144839
      Jiong Wang authored
      This patch implements code-gen for new JMP32 instructions on NFP.
      Reviewed-by: default avatarJakub Kicinski <jakub.kicinski@netronome.com>
      Signed-off-by: default avatarJiong Wang <jiong.wang@netronome.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      46144839
    • Jiong Wang's avatar
      s390: bpf: implement jitting of JMP32 · 626a5f66
      Jiong Wang authored
      This patch implements code-gen for new JMP32 instructions on s390.
      
      Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
      Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
      Signed-off-by: default avatarJiong Wang <jiong.wang@netronome.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      626a5f66
    • Jiong Wang's avatar
      ppc: bpf: implement jitting of JMP32 · 5f645996
      Jiong Wang authored
      This patch implements code-gen for new JMP32 instructions on ppc.
      
      For JMP32 | JSET, instruction encoding for PPC_RLWINM_DOT is added to check
      the result of ANDing low 32-bit of operands.
      
      Cc: Naveen N. Rao <naveen.n.rao@linux.ibm.com>
      Cc: Sandipan Das <sandipan@linux.ibm.com>
      Signed-off-by: default avatarJiong Wang <jiong.wang@netronome.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      5f645996
    • Jiong Wang's avatar
      arm: bpf: implement jitting of JMP32 · b85062ac
      Jiong Wang authored
      This patch implements code-gen for new JMP32 instructions on arm.
      
      For JSET, "ands" (AND with flags updated) is used, so corresponding
      encoding helper is added.
      
      Cc: Shubham Bansal <illusionist.neo@gmail.com>
      Signed-off-by: default avatarJiong Wang <jiong.wang@netronome.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      b85062ac
    • Jiong Wang's avatar
      arm64: bpf: implement jitting of JMP32 · 654b65a0
      Jiong Wang authored
      This patch implements code-gen for new JMP32 instructions on arm64.
      
      Cc: Daniel Borkmann <daniel@iogearbox.net>
      Cc: Alexei Starovoitov <ast@kernel.org>
      Cc: Zi Shen Lim <zlim.lnx@gmail.com>
      Signed-off-by: default avatarJiong Wang <jiong.wang@netronome.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      654b65a0
    • Jiong Wang's avatar
      x32: bpf: implement jitting of JMP32 · 69f827eb
      Jiong Wang authored
      This patch implements code-gen for new JMP32 instructions on x32.
      Also fixed several reverse xmas tree coding style issues as I am there.
      
      Cc: Wang YanQing <udknight@gmail.com>
      Signed-off-by: default avatarJiong Wang <jiong.wang@netronome.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      69f827eb
    • Jiong Wang's avatar
      x86_64: bpf: implement jitting of JMP32 · 3f5d6525
      Jiong Wang authored
      This patch implements code-gen for new JMP32 instructions on x86_64.
      
      Cc: Alexei Starovoitov <ast@kernel.org>
      Cc: Daniel Borkmann <daniel@iogearbox.net>
      Signed-off-by: default avatarJiong Wang <jiong.wang@netronome.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      3f5d6525
    • Jiong Wang's avatar
      bpf: JIT blinds support JMP32 · a7b76c88
      Jiong Wang authored
      This patch adds JIT blinds support for JMP32.
      
      Like BPF_JMP_REG/IMM, JMP32 version are needed for building raw bpf insn.
      They are added to both include/linux/filter.h and
      tools/include/linux/filter.h.
      Reviewed-by: default avatarJakub Kicinski <jakub.kicinski@netronome.com>
      Signed-off-by: default avatarJiong Wang <jiong.wang@netronome.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      a7b76c88
    • Jiong Wang's avatar
      bpf: interpreter support for JMP32 · 503a8865
      Jiong Wang authored
      This patch implements interpreting new JMP32 instructions.
      Reviewed-by: default avatarJakub Kicinski <jakub.kicinski@netronome.com>
      Signed-off-by: default avatarJiong Wang <jiong.wang@netronome.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      503a8865
    • Jiong Wang's avatar
      tools: bpftool: teach cfg code about JMP32 · df791dc1
      Jiong Wang authored
      The cfg code need to be aware of the new JMP32 instruction class so it
      could partition functions correctly.
      Reviewed-by: default avatarJakub Kicinski <jakub.kicinski@netronome.com>
      Signed-off-by: default avatarJiong Wang <jiong.wang@netronome.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      df791dc1
    • Jiong Wang's avatar
      bpf: disassembler support JMP32 · 56cbd82e
      Jiong Wang authored
      This patch teaches disassembler about JMP32. There are two places to
      update:
      
        - Class 0x6 now used by BPF_JMP32, not "unused".
      
        - BPF_JMP32 need to show comparison operands properly.
          The disassemble format is to add an extra "(32)" before the operands if
          it is a sub-register. A better disassemble format for both JMP32 and
          ALU32 just show the register prefix as "w" instead of "r", this is the
          format using by LLVM assembler.
      Reviewed-by: default avatarJakub Kicinski <jakub.kicinski@netronome.com>
      Signed-off-by: default avatarJiong Wang <jiong.wang@netronome.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      56cbd82e
    • Jiong Wang's avatar
      bpf: verifier support JMP32 · 092ed096
      Jiong Wang authored
      This patch teach verifier about the new BPF_JMP32 instruction class.
      Verifier need to treat it similar as the existing BPF_JMP class.
      A BPF_JMP32 insn needs to go through all checks that have been done on
      BPF_JMP.
      
      Also, verifier is doing runtime optimizations based on the extra info
      conditional jump instruction could offer, especially when the comparison is
      between constant and register that the value range of the register could be
      improved based on the comparison results. These code are updated
      accordingly.
      Acked-by: default avatarJakub Kicinski <jakub.kicinski@netronome.com>
      Signed-off-by: default avatarJiong Wang <jiong.wang@netronome.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      092ed096
    • Jiong Wang's avatar
      bpf: refactor verifier min/max code for condition jump · a72dafaf
      Jiong Wang authored
      The current min/max code does both signed and unsigned comparisons against
      the input argument "val" which is "u64" and there is explicit type casting
      when the comparison is signed.
      
      As we will need slightly more complexer type casting when JMP32 introduced,
      it is better to host the signed type casting. This makes the code more
      clean with ignorable runtime overhead.
      
      Also, code for J*GE/GT/LT/LE and JEQ/JNE are very similar, this patch
      combine them.
      
      The main purpose for this refactor is to make sure the min/max code will
      still be readable and with minimum code duplication after JMP32 introduced.
      Reviewed-by: default avatarJakub Kicinski <jakub.kicinski@netronome.com>
      Signed-off-by: default avatarJiong Wang <jiong.wang@netronome.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      a72dafaf
    • Jiong Wang's avatar
      bpf: allocate 0x06 to new eBPF instruction class JMP32 · d405c740
      Jiong Wang authored
      The new eBPF instruction class JMP32 uses the reserved class number 0x6.
      Kernel BPF ISA documentation updated accordingly.
      Reviewed-by: default avatarJakub Kicinski <jakub.kicinski@netronome.com>
      Signed-off-by: default avatarJiong Wang <jiong.wang@netronome.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      d405c740
  2. 25 Jan, 2019 5 commits
  3. 24 Jan, 2019 15 commits
  4. 23 Jan, 2019 3 commits