[PATCH] Yet another RCU documentation update

Update RCU documentation based on discussions and review of RCU-based tree
patches.  Add an introductory whatisRCU.txt file.

Signed-off-by: <paulmck@us.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>

Documentation/RCU/RTFP.txt

@@ -2,7 +2,8 @@ Read the F-ing Papers!
 
 
 This document describes RCU-related publications, and is followed by
-the corresponding bibtex entries.
+the corresponding bibtex entries.  A number of the publications may
+be found at http://www.rdrop.com/users/paulmck/RCU/.
 
 The first thing resembling RCU was published in 1980, when Kung and Lehman
 [Kung80] recommended use of a garbage collector to defer destruction
@@ -113,6 +114,10 @@ describing how to make RCU safe for soft-realtime applications [Sarma04c],
 and a paper describing SELinux performance with RCU [JamesMorris04b].
 
 
+2005 has seen further adaptation of RCU to realtime use, permitting
+preemption of RCU realtime critical sections [PaulMcKenney05a,
+PaulMcKenney05b].
+
 Bibtex Entries
 
 @article{Kung80
@@ -410,3 +415,32 @@ Oregon Health and Sciences University"
 \url{http://www.livejournal.com/users/james_morris/2153.html}
 [Viewed December 10, 2004]"
 }
+
+@unpublished{PaulMcKenney05a
+,Author="Paul E. McKenney"
+,Title="{[RFC]} {RCU} and {CONFIG\_PREEMPT\_RT} progress"
+,month="May"
+,year="2005"
+,note="Available:
+\url{http://lkml.org/lkml/2005/5/9/185}
+[Viewed May 13, 2005]"
+,annotation="
+	First publication of working lock-based deferred free patches
+	for the CONFIG_PREEMPT_RT environment.
+"
+}
+
+@conference{PaulMcKenney05b
+,Author="Paul E. McKenney and Dipankar Sarma"
+,Title="Towards Hard Realtime Response from the Linux Kernel on SMP Hardware"
+,Booktitle="linux.conf.au 2005"
+,month="April"
+,year="2005"
+,address="Canberra, Australia"
+,note="Available:
+\url{http://www.rdrop.com/users/paulmck/RCU/realtimeRCU.2005.04.23a.pdf}
+[Viewed May 13, 2005]"
+,annotation="
+	Realtime turns into making RCU yet more realtime friendly.
+"
+}

Documentation/RCU/UP.txt

@@ -8,7 +8,7 @@ is that since there is only one CPU, it should not be necessary to
 wait for anything else to get done, since there are no other CPUs for
 anything else to be happening on.  Although this approach will -sort- -of-
 work a surprising amount of the time, it is a very bad idea in general.
-This document presents two examples that demonstrate exactly how bad an
+This document presents three examples that demonstrate exactly how bad an
 idea this is.
 
 
@@ -26,6 +26,9 @@ from softirq, the list scan would find itself referencing a newly freed
 element B.  This situation can greatly decrease the life expectancy of
 your kernel.
 
+This same problem can occur if call_rcu() is invoked from a hardware
+interrupt handler.
+
 
 Example 2: Function-Call Fatality
 
@@ -44,8 +47,37 @@ its arguments would cause it to fail to make the fundamental guarantee
 underlying RCU, namely that call_rcu() defers invoking its arguments until
 all RCU read-side critical sections currently executing have completed.
 
-Quick Quiz: why is it -not- legal to invoke synchronize_rcu() in
-this case?
+Quick Quiz #1: why is it -not- legal to invoke synchronize_rcu() in
+	this case?
+
+
+Example 3: Death by Deadlock
+
+Suppose that call_rcu() is invoked while holding a lock, and that the
+callback function must acquire this same lock.  In this case, if
+call_rcu() were to directly invoke the callback, the result would
+be self-deadlock.
+
+In some cases, it would possible to restructure to code so that
+the call_rcu() is delayed until after the lock is released.  However,
+there are cases where this can be quite ugly:
+
+1.	If a number of items need to be passed to call_rcu() within
+	the same critical section, then the code would need to create
+	a list of them, then traverse the list once the lock was
+	released.
+
+2.	In some cases, the lock will be held across some kernel API,
+	so that delaying the call_rcu() until the lock is released
+	requires that the data item be passed up via a common API.
+	It is far better to guarantee that callbacks are invoked
+	with no locks held than to have to modify such APIs to allow
+	arbitrary data items to be passed back up through them.
+
+If call_rcu() directly invokes the callback, painful locking restrictions
+or API changes would be required.
+
+Quick Quiz #2: What locking restriction must RCU callbacks respect?
 
 
 Summary
@@ -53,12 +85,35 @@ Summary
 Permitting call_rcu() to immediately invoke its arguments or permitting
 synchronize_rcu() to immediately return breaks RCU, even on a UP system.
 So do not do it!  Even on a UP system, the RCU infrastructure -must-
-respect grace periods.
-
-
-Answer to Quick Quiz
-
-The calling function is scanning an RCU-protected linked list, and
-is therefore within an RCU read-side critical section.  Therefore,
-the called function has been invoked within an RCU read-side critical
-section, and is not permitted to block.
+respect grace periods, and -must- invoke callbacks from a known environment
+in which no locks are held.
+
+
+Answer to Quick Quiz #1:
+	Why is it -not- legal to invoke synchronize_rcu() in this case?
+
+	Because the calling function is scanning an RCU-protected linked
+	list, and is therefore within an RCU read-side critical section.
+	Therefore, the called function has been invoked within an RCU
+	read-side critical section, and is not permitted to block.
+
+Answer to Quick Quiz #2:
+	What locking restriction must RCU callbacks respect?
+
+	Any lock that is acquired within an RCU callback must be
+	acquired elsewhere using an _irq variant of the spinlock
+	primitive.  For example, if "mylock" is acquired by an
+	RCU callback, then a process-context acquisition of this
+	lock must use something like spin_lock_irqsave() to
+	acquire the lock.
+
+	If the process-context code were to simply use spin_lock(),
+	then, since RCU callbacks can be invoked from softirq context,
+	the callback might be called from a softirq that interrupted
+	the process-context critical section.  This would result in
+	self-deadlock.
+
+	This restriction might seem gratuitous, since very few RCU
+	callbacks acquire locks directly.  However, a great many RCU
+	callbacks do acquire locks -indirectly-, for example, via
+	the kfree() primitive.

Documentation/RCU/checklist.txt

@@ -43,6 +43,10 @@ over a rather long period of time, but improvements are always welcome!
 	rcu_read_lock_bh()) in the read-side critical sections,
 	and are also an excellent aid to readability.
 
+	As a rough rule of thumb, any dereference of an RCU-protected
+	pointer must be covered by rcu_read_lock() or rcu_read_lock_bh()
+	or by the appropriate update-side lock.
+
 3.	Does the update code tolerate concurrent accesses?
 
 	The whole point of RCU is to permit readers to run without
@@ -90,7 +94,11 @@ over a rather long period of time, but improvements are always welcome!
 
 		The rcu_dereference() primitive is used by the various
 		"_rcu()" list-traversal primitives, such as the
-		list_for_each_entry_rcu().
+		list_for_each_entry_rcu().  Note that it is perfectly
+		legal (if redundant) for update-side code to use
+		rcu_dereference() and the "_rcu()" list-traversal
+		primitives.  This is particularly useful in code
+		that is common to readers and updaters.
 
 	b.	If the list macros are being used, the list_add_tail_rcu()
 		and list_add_rcu() primitives must be used in order
@@ -150,16 +158,9 @@ over a rather long period of time, but improvements are always welcome!
 
 	Use of the _rcu() list-traversal primitives outside of an
 	RCU read-side critical section causes no harm other than
-	a slight performance degradation on Alpha CPUs and some
-	confusion on the part of people trying to read the code.
-
-	Another way of thinking of this is "If you are holding the
-	lock that prevents the data structure from changing, why do
-	you also need RCU-based protection?"  That said, there may
-	well be situations where use of the _rcu() list-traversal
-	primitives while the update-side lock is held results in
-	simpler and more maintainable code.  The jury is still out
-	on this question.
+	a slight performance degradation on Alpha CPUs.  It can
+	also be quite helpful in reducing code bloat when common
+	code is shared between readers and updaters.
 
 10.	Conversely, if you are in an RCU read-side critical section,
 	you -must- use the "_rcu()" variants of the list macros.

Documentation/RCU/rcu.txt

@@ -64,6 +64,54 @@ o	I hear that RCU is patented?  What is with that?
 	Of these, one was allowed to lapse by the assignee, and the
 	others have been contributed to the Linux kernel under GPL.
 
+o	I hear that RCU needs work in order to support realtime kernels?
+
+	Yes, work in progress.
+
 o	Where can I find more information on RCU?
 
 	See the RTFP.txt file in this directory.
+	Or point your browser at http://www.rdrop.com/users/paulmck/RCU/.
+
+o	What are all these files in this directory?
+
+
+	NMI-RCU.txt
+
+		Describes how to use RCU to implement dynamic
+		NMI handlers, which can be revectored on the fly,
+		without rebooting.
+
+	RTFP.txt
+
+		List of RCU-related publications and web sites.
+
+	UP.txt
+
+		Discussion of RCU usage in UP kernels.
+
+	arrayRCU.txt
+
+		Describes how to use RCU to protect arrays, with
+		resizeable arrays whose elements reference other
+		data structures being of the most interest.
+
+	checklist.txt
+
+		Lists things to check for when inspecting code that
+		uses RCU.
+
+	listRCU.txt
+
+		Describes how to use RCU to protect linked lists.
+		This is the simplest and most common use of RCU
+		in the Linux kernel.
+
+	rcu.txt
+
+		You are reading it!
+
+	whatisRCU.txt
+
+		Overview of how the RCU implementation works.  Along
+		the way, presents a conceptual view of RCU.