1. 05 Aug, 2021 1 commit
  2. 03 Aug, 2021 2 commits
    • Ilya Leoshkevich's avatar
      s390/ftrace: implement hotpatching · de5012b4
      Ilya Leoshkevich authored
      s390 allows hotpatching the mask of a conditional jump instruction.
      Make use of this feature in order to avoid the expensive stop_machine()
      call.
      
      The new trampolines are split in 3 stages:
      
      - A first stage is a 6-byte relative conditional long branch located at
        each function's entry point. Its offset always points to the second
        stage for the corresponding function, and its mask is either all 0s
        (ftrace off) or all 1s (ftrace on). The code for flipping the mask is
        borrowed from ftrace_{enable,disable}_ftrace_graph_caller. After
        flipping, ftrace_arch_code_modify_post_process() syncs with all the
        other CPUs by sending SIGPs.
      
      - Second stages for vmlinux are stored in a separate part of the .text
        section reserved by the linker script, and in dynamically allocated
        memory for modules. This prevents the icache pollution. The total
        size of second stages is about 1.5% of that of the kernel image.
      
        Putting second stages in the .bss section is possible and decreases
        the size of the non-compressed vmlinux, but splits the kernel 1:1
        mapping, which is a bad tradeoff.
      
        Each second stage contains a call to the third stage, a pointer to
        the part of the intercepted function right after the first stage, and
        a pointer to an interceptor function (e.g. ftrace_caller).
      
        Second stages are 8-byte aligned for the future direct calls
        implementation.
      
      - There are only two copies of the third stage: in the .text section
        for vmlinux and in dynamically allocated memory for modules. It can be
        an expoline, which is relatively large, so inlining it into each
        second stage is prohibitively expensive.
      
      As a result of this organization, phoronix-test-suite with ftrace off
      does not show any performance degradation.
      Suggested-by: default avatarSven Schnelle <svens@linux.ibm.com>
      Suggested-by: default avatarVasily Gorbik <gor@linux.ibm.com>
      Co-developed-by: default avatarHeiko Carstens <hca@linux.ibm.com>
      Signed-off-by: default avatarIlya Leoshkevich <iii@linux.ibm.com>
      Link: https://lore.kernel.org/r/20210728212546.128248-3-iii@linux.ibm.comSigned-off-by: default avatarHeiko Carstens <hca@linux.ibm.com>
      de5012b4
    • Ilya Leoshkevich's avatar
      ftrace: Introduce ftrace_need_init_nop() · 67ccddf8
      Ilya Leoshkevich authored
      Implementing live patching on s390 requires each function's prologue to
      contain a very special kind of nop, which gcc and clang don't generate.
      However, the current code assumes that if CC_USING_NOP_MCOUNT is
      defined, then whatever the compiler generates is good enough.
      
      Move the CC_USING_NOP_MCOUNT check into the new ftrace_need_init_nop()
      macro, that the architectures can override.
      
      An alternative solution is to disable using -mnop-mcount in the
      Makefile, however, this makes the build logic (even) more complicated
      and forces the arch-specific code to deal with the useless __fentry__
      symbol.
      Signed-off-by: default avatarIlya Leoshkevich <iii@linux.ibm.com>
      Reviewed-by: default avatarSteven Rostedt (VMware) <rostedt@goodmis.org>
      Link: https://lore.kernel.org/r/20210728212546.128248-2-iii@linux.ibm.comSigned-off-by: default avatarHeiko Carstens <hca@linux.ibm.com>
      67ccddf8
  3. 30 Jul, 2021 8 commits
  4. 27 Jul, 2021 29 commits