• Nicholas Piggin's avatar
    powerpc/64s/interrupt: handle MSR EE and RI in interrupt entry wrapper · ff0b0d6e
    Nicholas Piggin authored
    The mtmsrd to enable MSR[RI] can be combined with the mtmsrd to enable
    MSR[EE] in interrupt entry code, for those interrupts which enable EE.
    This helps performance of important synchronous interrupts (e.g., page
    faults).
    
    This is similar to what commit dd152f70 ("powerpc/64s: system call
    avoid setting MSR[RI] until we set MSR[EE]") does for system calls.
    
    Do this by enabling EE and RI together at the beginning of the entry
    wrapper if PACA_IRQ_HARD_DIS is clear, and only enabling RI if it is
    set.
    
    Asynchronous interrupts set PACA_IRQ_HARD_DIS, but synchronous ones
    leave it unchanged, so by default they always get EE=1 unless they have
    interrupted a caller that is hard disabled. When the sync interrupt
    later calls interrupt_cond_local_irq_enable(), it will not require
    another mtmsrd because MSR[EE] was already enabled here.
    
    This avoids one mtmsrd L=1 for synchronous interrupts on 64s, which
    saves about 20 cycles on POWER9. And for kernel-mode interrupts, both
    synchronous and asynchronous, this saves an additional 40 cycles due to
    the mtmsrd being moved ahead of mfspr SPRN_AMR, which prevents a SPR
    scoreboard stall.
    Signed-off-by: default avatarNicholas Piggin <npiggin@gmail.com>
    Signed-off-by: default avatarMichael Ellerman <mpe@ellerman.id.au>
    Link: https://lore.kernel.org/r/20210922145452.352571-3-npiggin@gmail.com
    ff0b0d6e
exceptions-64s.S 85.4 KB