• Chris Mason's avatar
    rds: recycle FMRs through lockless lists · 6fa70da6
    Chris Mason authored
    FRM allocation and recycling is performance critical and fairly lock
    intensive.  The current code has a per connection lock that all
    processes bang on and it becomes a major bottleneck on large systems.
    
    This changes things to use a number of cmpxchg based lists instead,
    allowing us to go through the whole FMR lifecycle without locking inside
    RDS.
    
    Zach Brown pointed out that our usage of cmpxchg for xlist removal is
    racey if someone manages to remove and add back an FMR struct into the list
    while another CPU can see the FMR's address at the head of the list.
    
    The second CPU might assume the list hasn't changed when in fact any
    number of operations might have happened in between the deletion and
    reinsertion.
    
    This commit maintains a per cpu count of CPUs that are currently
    in xlist removal, and establishes a grace period to make sure that
    nobody can see an entry we have just removed from the list.
    Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
    6fa70da6
ib_rdma.c 19.8 KB