• Huaisheng Ye's avatar
    dm writecache: optimize performance by sorting the blocks for writeback_all · 5229b489
    Huaisheng Ye authored
    During the process of writeback, the blocks, which have been placed in wbl.list
    for writeback soon, are partially ordered for the contiguous ones.
    
    When writeback_all has been set, for most cases, also by default, there will be
    a lot of blocks in pmem need to writeback at the same time.
    For this case, we could optimize the performance by sorting all blocks in
    wbl.list. writecache_writeback doesn't need to get blocks from the tail of
    wc->lru, whereas from the first rb_node from the rb_tree.
    
    The benefit is that, writecache_writeback doesn't need to have any cost to sort
    the blocks, because of all blocks are incremental originally in rb_tree.
    There will be a writecache_flush when writeback_all begins to work, that will
    eliminate duplicate blocks in cache by committed/uncommitted.
    
    Testing platform: Thinksystem SR630 with persistent memory.
    The cache comes from pmem, which has 1006MB size. The origin device is HDD, 2GB
    of which for using.
    
    Testing steps:
     1) dmsetup create mycache --table '0 4194304 writecache p /dev/sdb1 /dev/pmem4  4096 0'
     2) fio -filename=/dev/mapper/mycache -direct=1 -iodepth=20 -rw=randwrite
     -ioengine=libaio -bs=4k -loops=1  -size=2g -group_reporting -name=mytest1
     3) time dmsetup message /dev/mapper/mycache 0 flush
    
    Here is the results below,
    With the patch:
     # fio -filename=/dev/mapper/mycache -direct=1 -iodepth=20 -rw=randwrite
     -ioengine=libaio -bs=4k -loops=1  -size=2g -group_reporting -name=mytest1
       iops        : min= 1582, max=199470, avg=5305.94, stdev=21273.44, samples=197
     # time dmsetup message /dev/mapper/mycache 0 flush
    real	0m44.020s
    user	0m0.002s
    sys	0m0.003s
    
    Without the patch:
     # fio -filename=/dev/mapper/mycache -direct=1 -iodepth=20 -rw=randwrite
     -ioengine=libaio -bs=4k -loops=1  -size=2g -group_reporting -name=mytest1
       iops        : min= 1202, max=197650, avg=4968.67, stdev=20480.17, samples=211
     # time dmsetup message /dev/mapper/mycache 0 flush
    real	1m39.221s
    user	0m0.001s
    sys	0m0.003s
    
    I also have checked the data accuracy with this patch by making EXT4 filesystem
    on mycache, then mount it for checking md5 of files on that.
    The test result is positive, with this patch it could save more than half of time
    when writeback_all.
    Signed-off-by: default avatarHuaisheng Ye <yehs1@lenovo.com>
    Signed-off-by: default avatarMike Snitzer <snitzer@redhat.com>
    5229b489
dm-writecache.c 56.8 KB