• Chuck Lever's avatar
    svcrdma: Introduce local rdma_rw API helpers · f13193f5
    Chuck Lever authored
    The plan is to replace the local bespoke code that constructs and
    posts RDMA Read and Write Work Requests with calls to the rdma_rw
    API. This shares code with other RDMA-enabled ULPs that manages the
    gory details of buffer registration and posting Work Requests.
    
    Some design notes:
    
     o The structure of RPC-over-RDMA transport headers is flexible,
       allowing multiple segments per Reply with arbitrary alignment,
       each with a unique R_key. Write and Send WRs continue to be
       built and posted in separate code paths. However, one whole
       chunk (with one or more RDMA segments apiece) gets exactly
       one ib_post_send and one work completion.
    
     o svc_xprt reference counting is modified, since a chain of
       rdma_rw_ctx structs generates one completion, no matter how
       many Write WRs are posted.
    
     o The current code builds the transport header as it is construct-
       ing Write WRs. I've replaced that with marshaling of transport
       header data items in a separate step. This is because the exact
       structure of client-provided segments may not align with the
       components of the server's reply xdr_buf, or the pages in the
       page list. Thus parts of each client-provided segment may be
       written at different points in the send path.
    Signed-off-by: default avatarChuck Lever <chuck.lever@oracle.com>
    Signed-off-by: default avatarJ. Bruce Fields <bfields@redhat.com>
    f13193f5
svc_rdma_transport.c 37.3 KB