• Israel Rukshin's avatar
    IB/iser: Use IB_WR_REG_MR_INTEGRITY for PI handover · b76a4399
    Israel Rukshin authored
    Using this new API reduces iSER code complexity.
    It also reduces the maximum number of work requests per task and the need
    of dealing with multiple MRs (and their registrations and invalidations)
    per task. It is done by using a single WR and a special MR type
    (IB_MR_TYPE_INTEGRITY) for PI operation.
    
    The setup of the tested benchmark:
     - 2 servers with 24 cores (1 initiator and 1 target)
     - 24 target sessions with 1 LUN each
     - ramdisk backstore
     - PI active
    
    Performance results running fio (24 jobs, 128 iodepth) using
    write_generate=0 and read_verify=0 (w/w.o patch):
    
    bs      IOPS(read)        IOPS(write)
    ----    ----------        ----------
    512     1236.6K/1164.3K   1357.2K/1332.8K
    1k      1196.5K/1163.8K   1348.4K/1262.7K
    2k      1016.7K/921950    1003.7K/931230
    4k      662728/600545     595423/501513
    8k      385954/384345     333775/277090
    16k     222864/222820     170317/170671
    32k     116869/114896     82331/82244
    64k     55205/54931       40264/40021
    
    Using write_generate=1 and read_verify=1 (w/w.o patch):
    
    bs      IOPS(read)        IOPS(write)
    ----    ----------        ----------
    512     1090.1K/1030.9K   1303.9K/1101.4K
    1k      1057.7K/904583    1318.4K/988085
    2k      965226/638799     1008.6K/692514
    4k      555479/410151     542414/414517
    8k      298675/224964     264729/237508
    16k     133485/122481     164625/138647
    32k     74329/67615       80143/78743
    64k     35716/35519       39294/37334
    
    We get performance improvement at all block sizes.
    The most significant improvement is when writing 4k bs (almost 30% more
    iops).
    Signed-off-by: default avatarIsrael Rukshin <israelr@mellanox.com>
    Reviewed-by: default avatarMax Gurtovoy <maxg@mellanox.com>
    Reviewed-by: default avatarChristoph Hellwig <hch@lst.de>
    Reviewed-by: default avatarSagi Grimberg <sagi@grimberg.me>
    Signed-off-by: default avatarJason Gunthorpe <jgg@mellanox.com>
    b76a4399
iser_initiator.c 21.7 KB