• Xiaoguang Wang's avatar
    io_uring: avoid whole io_wq_work copy for requests completed inline · 7cdaf587
    Xiaoguang Wang authored
    If requests can be submitted and completed inline, we don't need to
    initialize whole io_wq_work in io_init_req(), which is an expensive
    operation, add a new 'REQ_F_WORK_INITIALIZED' to determine whether
    io_wq_work is initialized and add a helper io_req_init_async(), users
    must call io_req_init_async() for the first time touching any members
    of io_wq_work.
    
    I use /dev/nullb0 to evaluate performance improvement in my physical
    machine:
      modprobe null_blk nr_devices=1 completion_nsec=0
      sudo taskset -c 60 fio  -name=fiotest -filename=/dev/nullb0 -iodepth=128
      -thread -rw=read -ioengine=io_uring -direct=1 -bs=4k -size=100G -numjobs=1
      -time_based -runtime=120
    
    before this patch:
    Run status group 0 (all jobs):
       READ: bw=724MiB/s (759MB/s), 724MiB/s-724MiB/s (759MB/s-759MB/s),
       io=84.8GiB (91.1GB), run=120001-120001msec
    
    With this patch:
    Run status group 0 (all jobs):
       READ: bw=761MiB/s (798MB/s), 761MiB/s-761MiB/s (798MB/s-798MB/s),
       io=89.2GiB (95.8GB), run=120001-120001msec
    
    About 5% improvement.
    Signed-off-by: default avatarXiaoguang Wang <xiaoguang.wang@linux.alibaba.com>
    Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
    7cdaf587
io_uring.c 192 KB