• Roman Penyaev's avatar
    epoll: make sure all elements in ready list are in FIFO order · c141175d
    Roman Penyaev authored
    Patch series "use rwlock in order to reduce ep_poll_callback()
    contention", v3.
    
    The last patch targets the contention problem in ep_poll_callback(),
    which can be very well reproduced by generating events (write to pipe or
    eventfd) from many threads, while consumer thread does polling.
    
    The following are some microbenchmark results based on the test [1]
    which starts threads which generate N events each.  The test ends when
    all events are successfully fetched by the poller thread:
    
     spinlock
     ========
    
     threads  events/ms  run-time ms
           8       6402        12495
          16       7045        22709
          32       7395        43268
    
     rwlock + xchg
     =============
    
     threads  events/ms  run-time ms
           8      10038         7969
          16      12178        13138
          32      13223        24199
    
    According to the results bandwidth of delivered events is significantly
    increased, thus execution time is reduced.
    
    This patch (of 4):
    
    All coming events are stored in FIFO order and this is also should be
    applicable to ->ovflist, which originally is stack, i.e.  LIFO.
    
    Thus to keep correct FIFO order ->ovflist should reversed by adding
    elements to the head of the read list but not to the tail.
    
    Link: http://lkml.kernel.org/r/20190103150104.17128-2-rpenyaev@suse.deSigned-off-by: default avatarRoman Penyaev <rpenyaev@suse.de>
    Reviewed-by: default avatarDavidlohr Bueso <dbueso@suse.de>
    Cc: Jason Baron <jbaron@akamai.com>
    Cc: Al Viro <viro@zeniv.linux.org.uk>
    Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    c141175d
eventpoll.c 61.9 KB