• Ming Lei's avatar
    dm: support bio polling · b99fdcdc
    Ming Lei authored
    Support bio polling (REQ_POLLED) in the following approach:
    
    1) only support io polling on normal READ/WRITE, and other abnormal IOs
    still fallback to IRQ mode, so the target io (and DM's clone bio) is
    exactly inside the dm io.
    
    2) hold one refcnt on io->io_count after submitting this dm bio with
    REQ_POLLED
    
    3) support dm native bio splitting, any dm io instance associated with
    current bio will be added into one list which head is bio->bi_private
    which will be recovered before ending this bio
    
    4) implement .poll_bio() callback, call bio_poll() on the single target
    bio inside the dm io which is retrieved via bio->bi_bio_drv_data; call
    dm_io_dec_pending() after the target io is done in .poll_bio()
    
    5) enable QUEUE_FLAG_POLL if all underlying queues enable QUEUE_FLAG_POLL,
    which is based on Jeffle's previous patch.
    
    These changes are good for a 30-35% IOPS improvement for polled IO.
    
    For detailed test results please see (Jens, thanks for testing!):
    https://listman.redhat.com/archives/dm-devel/2022-March/049868.html
    or https://marc.info/?l=linux-block&m=164684246214700&w=2Tested-by: default avatarJens Axboe <axboe@kernel.dk>
    Signed-off-by: default avatarMing Lei <ming.lei@redhat.com>
    Signed-off-by: default avatarMike Snitzer <snitzer@redhat.com>
    b99fdcdc
dm-table.c 53.5 KB