• Chuck Lever's avatar
    xprtrdma: Allocate MRs on demand · e2ac236c
    Chuck Lever authored
    Frequent MR list exhaustion can impact I/O throughput, so enough MRs
    are always created during transport set-up to prevent running out.
    This means more MRs are created than most workloads need.
    
    Commit 94f58c58 ("xprtrdma: Allow Read list and Reply chunk
    simultaneously") introduced support for sending two chunk lists per
    RPC, which consumes more MRs per RPC.
    
    Instead of trying to provision more MRs, introduce a mechanism for
    allocating MRs on demand. A few MRs are allocated during transport
    set-up to kick things off.
    
    This significantly reduces the average number of MRs per transport
    while allowing the MR count to grow for workloads or devices that
    need more MRs.
    
    FRWR with mlx4 allocated almost 400 MRs per transport before this
    patch. Now it starts with 32.
    Signed-off-by: default avatarChuck Lever <chuck.lever@oracle.com>
    Tested-by: default avatarSteve Wise <swise@opengridcomputing.com>
    Signed-off-by: default avatarAnna Schumaker <Anna.Schumaker@Netapp.com>
    e2ac236c
transport.c 22 KB