• Alexander Duyck's avatar
    arch: Add lightweight memory barriers dma_rmb() and dma_wmb() · 1077fa36
    Alexander Duyck authored
    There are a number of situations where the mandatory barriers rmb() and
    wmb() are used to order memory/memory operations in the device drivers
    and those barriers are much heavier than they actually need to be.  For
    example in the case of PowerPC wmb() calls the heavy-weight sync
    instruction when for coherent memory operations all that is really needed
    is an lsync or eieio instruction.
    
    This commit adds a coherent only version of the mandatory memory barriers
    rmb() and wmb().  In most cases this should result in the barrier being the
    same as the SMP barriers for the SMP case, however in some cases we use a
    barrier that is somewhere in between rmb() and smp_rmb().  For example on
    ARM the rmb barriers break down as follows:
    
      Barrier   Call     Explanation
      --------- -------- ----------------------------------
      rmb()     dsb()    Data synchronization barrier - system
      dma_rmb() dmb(osh) data memory barrier - outer sharable
      smp_rmb() dmb(ish) data memory barrier - inner sharable
    
    These new barriers are not as safe as the standard rmb() and wmb().
    Specifically they do not guarantee ordering between coherent and incoherent
    memories.  The primary use case for these would be to enforce ordering of
    reads and writes when accessing coherent memory that is shared between the
    CPU and a device.
    
    It may also be noted that there is no dma_mb().  Most architectures don't
    provide a good mechanism for performing a coherent only full barrier without
    resorting to the same mechanism used in mb().  As such there isn't much to
    be gained in trying to define such a function.
    
    Cc: Frederic Weisbecker <fweisbec@gmail.com>
    Cc: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
    Cc: Michael Ellerman <michael@ellerman.id.au>
    Cc: Michael Neuling <mikey@neuling.org>
    Cc: Russell King <linux@arm.linux.org.uk>
    Cc: Geert Uytterhoeven <geert@linux-m68k.org>
    Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
    Cc: Tony Luck <tony.luck@intel.com>
    Cc: Oleg Nesterov <oleg@redhat.com>
    Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Ingo Molnar <mingo@kernel.org>
    Cc: David Miller <davem@davemloft.net>
    Acked-by: default avatarBenjamin Herrenschmidt <benh@kernel.crashing.org>
    Acked-by: default avatarWill Deacon <will.deacon@arm.com>
    Signed-off-by: default avatarAlexander Duyck <alexander.h.duyck@redhat.com>
    Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    1077fa36
barrier.h 3.65 KB