• Jens Axboe's avatar
    io_uring/net: improve the usercopy for sendmsg/recvmsg · 792060de
    Jens Axboe authored
    We're spending a considerable amount of the sendmsg/recvmsg time just
    copying in the message header. And for provided buffers, the known
    single entry iovec.
    
    Be a bit smarter about it and enable/disable user access around our
    copying. In a test case that does both sendmsg and recvmsg, the
    runtime before this change (averaged over multiple runs, very stable
    times however):
    
    Kernel		Time		Diff
    ====================================
    -git		4720 usec
    -git+commit	4311 usec	-8.7%
    
    and looking at a profile diff, we see the following:
    
    0.25%     +9.33%  [kernel.kallsyms]     [k] _copy_from_user
    4.47%     -3.32%  [kernel.kallsyms]     [k] __io_msg_copy_hdr.constprop.0
    
    where we drop more than 9% of _copy_from_user() time, and consequently
    add time to __io_msg_copy_hdr() where the copies are now attributed to,
    but with a net win of 6%.
    
    In comparison, the same test case with send/recv runs in 3745 usec, which
    is (expectedly) still quite a bit faster. But at least sendmsg/recvmsg is
    now only ~13% slower, where it was ~21% slower before.
    Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
    792060de
net.c 40 KB