• Jens Axboe's avatar
    io_uring: ensure RCU callback ordering with rcu_barrier() · 805b13ad
    Jens Axboe authored
    After more careful studying, Paul informs me that we cannot rely on
    ordering of RCU callbacks in the way that the the tagged commit did.
    The current construct looks like this:
    
    	void C(struct rcu_head *rhp)
    	{
    		do_something(rhp);
    		call_rcu(&p->rh, B);
    	}
    
    	call_rcu(&p->rh, A);
    	call_rcu(&p->rh, C);
    
    and we're relying on ordering between A and B, which isn't guaranteed.
    Make this explicit instead, and have a work item issue the rcu_barrier()
    to ensure that A has run before we manually execute B.
    
    While thorough testing never showed this issue, it's dependent on the
    per-cpu load in terms of RCU callbacks. The updated method simplifies
    the code as well, and eliminates the need to maintain an rcu_head in
    the fileset data.
    
    Fixes: c1e2148f ("io_uring: free fixed_file_data after RCU grace period")
    Reported-by: default avatarPaul E. McKenney <paulmck@kernel.org>
    Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
    805b13ad
io_uring.c 170 KB