• Chuck Lever's avatar
    xprtrdma: Move Receive posting to Receive handler · 7c8d9e7c
    Chuck Lever authored
    Receive completion and Reply handling are done by a BOUND
    workqueue, meaning they run on only one CPU.
    
    Posting receives is currently done in the send_request path, which
    on large systems is typically done on a different CPU than the one
    handling Receive completions. This results in movement of
    Receive-related cachelines between the sending and receiving CPUs.
    
    More importantly, it means that currently Receives are posted while
    the transport's write lock is held, which is unnecessary and costly.
    
    Finally, allocation of Receive buffers is performed on-demand in
    the Receive completion handler. This helps guarantee that they are
    allocated on the same NUMA node as the CPU that handles Receive
    completions.
    Signed-off-by: default avatarChuck Lever <chuck.lever@oracle.com>
    Signed-off-by: default avatarAnna Schumaker <Anna.Schumaker@Netapp.com>
    7c8d9e7c
rpcrdma.h 19.8 KB