• Eric Biggers's avatar
    statx: optimize copy of struct statx to userspace · 64bd7204
    Eric Biggers authored
    I found that statx() was significantly slower than stat().  As a
    microbenchmark, I compared 10,000,000 invocations of fstat() on a tmpfs
    file to the same with statx() passed a NULL path:
    
    	$ time ./stat_benchmark
    
    	real	0m1.464s
    	user	0m0.275s
    	sys	0m1.187s
    
    	$ time ./statx_benchmark
    
    	real	0m5.530s
    	user	0m0.281s
    	sys	0m5.247s
    
    statx is expected to be a little slower than stat because struct statx
    is larger than struct stat, but not by *that* much.  It turns out that
    most of the overhead was in copying struct statx to userspace, mostly in
    all the stac/clac instructions that got generated for each __put_user()
    call.  (This was on x86_64, but some other architectures, e.g. arm64,
    have something similar now too.)
    
    stat() instead initializes its struct on the stack and copies it to
    userspace with a single call to copy_to_user().  This turns out to be
    much faster, and changing statx to do this makes it almost as fast as
    stat:
    
    	$ time ./statx_benchmark
    
    	real	0m1.624s
    	user	0m0.270s
    	sys	0m1.354s
    
    For zeroing the reserved fields, start by zeroing the full struct with
    memset.  This makes it clear that every byte copied to userspace is
    initialized, even implicit padding bytes (though there are none
    currently).  In the scenarios I tested, it also performed the same as a
    designated initializer.  Manually initializing each field was still
    slightly faster, but would have been more error-prone and less
    verifiable.
    
    Also rename statx_set_result() to cp_statx() for consistency with
    cp_old_stat() et al., and make it noinline so that struct statx doesn't
    add to the stack usage during the main portion of the syscall execution.
    Signed-off-by: default avatarEric Biggers <ebiggers@google.com>
    Signed-off-by: default avatarDavid Howells <dhowells@redhat.com>
    Signed-off-by: default avatarAl Viro <viro@zeniv.linux.org.uk>
    64bd7204
stat.c 16.9 KB