• Zqiang's avatar
    rcu/kvfree: Use consistent krcp when growing kfree_rcu() page cache · 309a4316
    Zqiang authored
    The add_ptr_to_bulk_krc_lock() function is invoked to allocate a new
    kfree_rcu() page, also known as a kvfree_rcu_bulk_data structure.
    The kfree_rcu_cpu structure's lock is used to protect this operation,
    except that this lock must be momentarily dropped when allocating memory.
    It is clearly important that the lock that is reacquired be the same
    lock that was acquired initially via krc_this_cpu_lock().
    
    Unfortunately, this same krc_this_cpu_lock() function is used to
    re-acquire this lock, and if the task migrated to some other CPU during
    the memory allocation, this will result in the kvfree_rcu_bulk_data
    structure being added to the wrong CPU's kfree_rcu_cpu structure.
    
    This commit therefore replaces that second call to krc_this_cpu_lock()
    with raw_spin_lock_irqsave() in order to explicitly acquire the lock on
    the correct kfree_rcu_cpu structure, thus keeping things straight even
    when the task migrates.
    Signed-off-by: default avatarZqiang <qiang1.zhang@intel.com>
    Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
    309a4316
tree.c 162 KB