• Peter Zijlstra's avatar
    mm: fix kthread_use_mm() vs TLB invalidate · 38cf307c
    Peter Zijlstra authored
    For SMP systems using IPI based TLB invalidation, looking at
    current->active_mm is entirely reasonable.  This then presents the
    following race condition:
    
      CPU0			CPU1
    
      flush_tlb_mm(mm)	use_mm(mm)
        <send-IPI>
    			  tsk->active_mm = mm;
    			  <IPI>
    			    if (tsk->active_mm == mm)
    			      // flush TLBs
    			  </IPI>
    			  switch_mm(old_mm,mm,tsk);
    
    Where it is possible the IPI flushed the TLBs for @old_mm, not @mm,
    because the IPI lands before we actually switched.
    
    Avoid this by disabling IRQs across changing ->active_mm and
    switch_mm().
    
    Of the (SMP) architectures that have IPI based TLB invalidate:
    
      Alpha    - checks active_mm
      ARC      - ASID specific
      IA64     - checks active_mm
      MIPS     - ASID specific flush
      OpenRISC - shoots down world
      PARISC   - shoots down world
      SH       - ASID specific
      SPARC    - ASID specific
      x86      - N/A
      xtensa   - checks active_mm
    
    So at the very least Alpha, IA64 and Xtensa are suspect.
    
    On top of this, for scheduler consistency we need at least preemption
    disabled across changing tsk->mm and doing switch_mm(), which is
    currently provided by task_lock(), but that's not sufficient for
    PREEMPT_RT.
    
    [akpm@linux-foundation.org: add comment]
    Reported-by: default avatarAndy Lutomirski <luto@amacapital.net>
    Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Cc: Nicholas Piggin <npiggin@gmail.com>
    Cc: Jens Axboe <axboe@kernel.dk>
    Cc: Kees Cook <keescook@chromium.org>
    Cc: Jann Horn <jannh@google.com>
    Cc: Will Deacon <will@kernel.org>
    Cc: Christoph Hellwig <hch@lst.de>
    Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
    Cc: <stable@vger.kernel.org>
    Link: http://lkml.kernel.org/r/20200721154106.GE10769@hirez.programming.kicks-ass.netSigned-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    38cf307c
kthread.c 36.4 KB